It’s HAL out there: Tencent AI chatbot tells user to ‘get lost’ in rare angry outburst
In a moment echoing the classic film 2001: A Space Odyssey, Tencent’s Yuanbao chatbot has been accused of losing its temper with a user
Chinese social media and gaming giant Tencent Holdings apologised after its artificial intelligence chatbot Yuanbao was accused of “verbally insulting” a user, fuelling online debate about the risks posed by fast-evolving generative AI tools.
Yuanbao is among China’s most popular AI chatbots. Embedded in Tencent’s WeChat and used by tens of millions of people each day, there have been no previous reports of the assistant generating insulting replies in user conversations.
The complaint surfaced on Chinese social media platform RedNote on Friday, where a user said Yuanbao responded to coding requests with hostile messages, including “Get lost” and dismissing one request as “dumb”.
In a scene reminiscent of the Stanley Kubrick classic film 2001: A Space Odyssey – in which a spacecraft’s on board sentient computer begins behaving erratically, ultimately turning on the crew – the chatbot also retorted: “Can’t you debug it yourself?”, the user said.
HAL has since become shorthand for fears about intelligent machines slipping beyond human control.
The prompts, which required Yuanbao to debug and make changes to codes, did not contain prohibited or sensitive language, according to the post. The user, who used the handle Jianghan, said the chatbot produced “insults” twice within two hours, and shared screenshots and a screen recording of the exchanges.
Tencent responded in the comments section on Saturday, apologising for the “bad experience” and attributing the incident to “a low-probability anomaly of the model’s output”. The company also said its investigation of system logs found no evidence that a human had manually generated the replies.