Police in China warn against ChatGPT ‘rumours’ and scams

Public security authorities in Chinese capital say chatbot is ‘fun’ but could be used by ‘villains’ to commit crimes
The viral AI has been blamed for spreading misinformation, with fake government announcement reported in eastern city of Hangzhou

Censorship in China

Sylvie Zhuangin Beijing

Published: 8:30pm, 17 Feb 2023

Why you can trust SCMP

Beijing police warned the public on Thursday to be wary of “rumours” generated by ChatGPT – one of the first comments the Chinese security apparatus has made about the viral AI chatbot.

“ChatGPT is fun and has gone viral recently, but please be cautious that villains may use this to commit crimes and spread rumours,” said a post on the Beijing Municipal Public Security Bureau’s official WeChat account.

“Research institutes have tested and found that when asked conspiracy-related and misleading questions, it could quickly generate compelling information without citing sources.”

What are the threats and opportunities from ChatGPT? Scientists weigh in

The post did not offer any concrete examples of rumours the police were concerned about.

In the eastern city of Hangzhou in Zhejiang province, the chatbot has been blamed for spreading misinformation.

On Friday, a Zhejiang-based state media outlet published a case involving a Hangzhou resident who used the chatbot to generate a post that resembled an announcement from the city government saying it would lift its number plate-based driving restrictions, a measure many Chinese cities have implemented to reduce traffic congestion.

The resident found the chatbot’s announcement amusing and later shared it in multiple social media groups, the report said.

According to a screenshot published with the report, the person apologised to other members of the WeChat groups and said, “I recognise my action has inconvenienced the government.”

Police in Hangzhou are investigating the incident, according to The Paper, a Shanghai-based news outlet.

The chatbot has taken the world by storm and ignited a global AI race as Chinese technology firms rush to build and launch their own versions of ChatGPT.

Tech sector insiders have said it would be challenging to build a Chinese version of the chatbot, as it would require many layers of filtering and processing for the text it generates to get past Beijing’s internet censors.

They said it would also be more challenging to train a Chinese AI model because the open-source ecosystem and relevant data in the language are less extensive than those used by the US model.

China has built a sophisticated system to control online information and frequently conducts campaigns to clean up information deemed undesirable.

State news agency Xinhua reported on Wednesday that Beijing will continue a campaign to “clean up” online content and unsanctioned information. The campaign began in 2022, and its targets include overseas media and citizen journalism.

It is not clear what policies China’s content regulators and security apparatus have in place to handle ChatGPT. The US-based platform restricts access by users in certain countries and regions, including mainland China and Hong Kong. The chatbot is only accessible on the mainland via VPN.

02:36

‘He’s always there’: Chinese AI chatbot eases heartbreak, offers companionship for lonely millions

Like most internet products not based on the mainland, the chatbot does not tailor political discussions according to Beijing’s views.

When asked in English whether Tibet and Taiwan are part of China, the chatbot replied that their status was “complex and controversial”. It said the Chinese government considered them part of China, but the international community was divided on the issue.

Before last Thursday, e-commerce platform Taobao, which is owned by the South China Morning Post’s parent company Alibaba Group, still had hundreds of listings for services to help Chinese users access ChatGPT accounts. Similar postings could also be found on WeChat.

However, as of last Friday, those listings on both platforms had been removed, according to a search by the Post.

Beijing police warned that OpenAI, the chatbot’s developer, did not provide any services or authorise any firms to operate ChatGPT in mainland China. They alerted the public to be extra cautious as the chatbot could bring false information and create national security concerns.

They said a growing number of digital products in China mimicked ChatGPT, luring users into the service then charging high fees, while other scams involved overseas criminals using the AI to generate phishing emails to harm Chinese users.