Invader has visited our city twice before, in 2001 (19 works) and 2002 (six works), but the recent invasion was his largest ever (he calls the 48 new works "a very good score"). It was followed, however, by the most comprehensive take-down of his work ever.

"I was alerted on social networks through pictures and comments that government workers were removing some of the artworks displayed in the streets. Having invaded more than 60 cities around the world, I have never faced a situation where a public authority would systematically and rapidly remove the art from the streets," he says from his home in Paris.

"I fully understand that having my work damaged, stolen or removed is an inherent risk with displaying contemporary art in an urban environment, but it looks like 80 per cent of the pieces have been removed in less than two months. This wipe-out is part of a planned process from the local authorities, and this is really new for me."

Hong Kong has some form here. In 2009 Invader's fellow French artist, Christophe Schwarz, aka Zevs, was detained and threatened with damages after dripping easily removed water-based paint down a Chanel advertising logo in Central; while two years later, during the detention by mainland authorities of dissident artist Ai Weiwei, a rash of artworks supporting him appeared on Hong Kong walls - and were swiftly removed.

The issue is a vexed one for any city. Street art, graffiti, culture, vandalism: part of the issue is the terms of engagement one chooses to use. It might seem heavy-handed of the government to just tear works down, but it raises some tricky questions: if we're asking when graffiti becomes street art, then we're effectively asking what art is.

This is a weighty question for a government to decide, and one with a huge bedrock of cultural factors lying beneath it, to do with the way in which art is appreciated and valued here, and attitudes towards public expression.

"It's the same thing in every city: there's a certain amount of embracing it and a certain amount of resistance," says Sarah Ouellette, who last year organised the first Hong Kong edition of "Secret Walls", a street-art event in which artists face off in battle-like showdowns. "It's true in London, in New York and in Hong Kong. I lived in Melbourne and it's the same there; even though it's widely embraced and encouraged, it's still regularly removed. It's just an extra issue in Hong Kong because it's all so new here."

The novelty of street art is what's flummoxing the authorities, says Marlon Ge, a Hong Kong-based curator with a particular interest in street art. "The authorities do not understand it and are simply trying to discourage artists from what they consider vandalism."

Invader says usually about 10 per cent of his pieces are removed, through a combination of theft, breakage, destruction or refitting of the wall. This time, however, the removal has been so thorough that it's even affected his work from previous invasions. He says he was pleased to see during his recent visit that some of it was still in situ.

"I was, however, ulcerated to discover online that one of the first-wave pieces was removed a few days ago. It has never disturbed anyone, in particular the local authorities, for 12 years until now," he says.

"I am of course very saddened and affected by these removal actions. It is painful because I really consider that my approach of displaying the pieces in the street is a gift to the city and its citizens. It is a way for me to enhance people's everyday life. They don't need to go to museums or art galleries; they can just look up on the walls and maybe be touched by my 'urban acupuncture'."

The local authority responsible for the removal is the Highways Department. "The Highways Department is a works department," says a spokeswoman. "We are responsible for the maintenance of public roads and associated facilities. Any unauthorised materials or graffiti on public facilities are unlawful. If we identify any … during the course of inspections or through investigations of complaints that involve safety concerns, we will arrange for immediate removal."

Invader says police forces are usually hostile to "people scratching walls in the street"; he was most recently arrested in November in New York, which he's invaded five times. The Hong Kong police, by contrast, showed discretion. They discovered him working in Central but declined to arrest him once they'd taken a look at the piece and decided they liked it.

"I actually think that Hong Kong is one of the easiest cities in the world for a street artist or graffiti writer to tag or put up work," says Ge. "The police in Hong Kong are not as strict or as violent as in other cities. They consider graffiti a very minor issue. Many graffiti writers who come to Hong Kong find it very easy to tag here."

And Hong Kong is happy to preserve graffiti works under certain circumstances. The Highways Department says it has been told by the Home Affairs Bureau, which is responsible for cultural heritage and development, not to remove the remaining calligraphy work of Tsang Tsou-choi, known as the King of Kowloon.

Unfortunately, it had already removed most of it. And this is the problem: the government says it wants to promote culture, and while it is spending billions on projects such as the West Kowloon Cultural District, and in installing artworks in government buildings, parks, MTR stations and other public spaces, there's little by way of a mechanism to prevent works given free to the city by a talented artist being automatically removed.

"Regarding graffiti painting in public places, we note that there are different views in the community," a Home Affairs Bureau spokeswoman says. "Relevant government departments will handle such cases in accordance with their established procedures and the circumstances. The case of Tsang Tsou-choi has its own uniqueness to Hong Kong society, and is different from cases of graffiti painting in general."

This might be somewhat disingenuous: if Tsang were painting today, his work would be removed the same as anyone else's; when he was working, "graffiti painting in general" was exactly how his work was considered. Tsang's work is only "different" because of the benefit of hindsight - after he became a cultural cause celebre.

Ge says Hong Kong needs its own "Banksy moment": when a local artist becomes as famous as the British graffiti provocateur, views of street art in the city might well change. "I don't think any city has got it right; however, you do see some cities like London, which recognises Banksy's fame and protects his public work from being removed or vandalised. His public art pieces serve as tourist attractions to a certain extent, hence London's interest in protecting them."

Still, Invader says he loves Hong Kong. "Its people gave me a great welcome and it was a real pleasure to spend several weeks rediscovering the city: its heritage, its futurism and its dynamism.

"To look at art in the street is totally free, unlike advertising, for example. It doesn't bring any direct money to the government or big companies. If that were the case, maybe some people would be less hostile," he says.

"What they don't understand is that art is an indestructible, key element of human history. To the Hong Kong authorities, in case they intend to wipe out the entire invasion colony, I only ask: what message would you send to your citizens? What modern cultural heritage do you want to leave them? What is the real place of art in your beautiful city?"