Facebook owner Meta’s AI speech translation tool for Hokkien language draws interest in China

The Facebook owner says it has developed an artificial intelligence system for translation between spoken English and Hokkien
Hokkien is a primarily oral language widely spoken in parts of mainland China, Taiwan and Chinese diaspora communities across Southeast Asia

Artificial intelligence

Xinmei Shen

Published: 11:00am, 22 Oct, 2022

Why you can trust SCMP

Facebook owner Meta Platforms’ new real-time speech translation system for Hokkien, a dialect widely spoken in the southeastern Chinese province of Fujian, the self-ruled island of Taiwan and Chinese diaspora communities across Southeast Asia, has garnered strong interest on the mainland, where the social media platform is blocked.

The artificial intelligence (AI) system, which can translate between spoken Hokkien and English, is part of Meta’s efforts to “break down language barriers in both the physical world and the metaverse”, the US tech giant said this week in a blog post.

Existing AI-powered speech translation systems mostly focus on written languages and rely on transcriptions to train their AI models. Hokkien, however, is a primarily oral language that lacks a standard written form, according to Meta.

Another challenge is that there are far fewer Hokkien speakers than, say, English or Spanish speakers, which means there is insufficient training data available.

The difficulty of speech translation is illustrated earlier this month, when Douyin, the Chinese version of ByteDance-owned short video app TikTok, stirred controversy after it cut off the live-streaming sessions of several Cantonese-speaking influencers because its system could not recognise the Chinese dialect.

Meta’s speech-to-speech translation system overcame the problem by first translating English to Mandarin text, and then converting it into Hokkien, and vice versa, before adding that into training data.

When performing real-time translation, the system converts Hokkien into units of sounds and generates waveforms from them. The waveforms are then converted into Mandarin text, and eventually translated into sound units of English.

A busy intersection at a shopping district in Taipei, Taiwan, where Hokkien is widely spoken. Photo: EPA-EFE

The system, which Meta said is the first of its kind, stoked curiosity among netizens in mainland China, where a video showing Meta CEO Mark Zuckerberg and AI researcher Chen Peng-Jen using the tool to talk to each other in English and Hokkien, respectively, was shared by several local media outlets on Chinese social media platforms.

“Impressive. WeChat doesn’t even support voice input in Hokkien,” wrote one user of the Tencent Holdings super app who is shown as being based in Fujian province.

Some mainland internet users also encouraged Meta to take on the bigger challenge of developing a translation system for the local vernacular of the Wenzhou city in eastern Zhejiang province, which is famously difficult for even native Chinese speakers to grasp.

Other online commenters, however, see Meta’s new translation system as a potential threat to China amid intensifying tensions with the US, fearing that the technology could become a useful tool for the US government and military.

Hokkien, also known as Taiwanese Minnan or Hoklo, is the native language of many Taiwanese citizens whose families originated from Fujian, and is strongly associated with Taiwan’s pro-independence movement.

While Meta has made its Hokkien translation system open source in the hope that others would use it for more languages, the AI model is “still a work in progress” and can translate only one full sentence at a time, the company wrote.