Chinese generative AI developers rush to upgrade chat bots to handle super long texts
- The move follows Google, which recently unveiled the latest version of its Gemini LLM that can handle up to 1 million tokens, or roughly 700,000 English words
- Last week, generative AI start-up Moonshot AI announced a major update of its Kimi chat bot that can handle up to 2 million Chinese characters in a single prompt
China’s biggest generative artificial intelligence (AI) developers, including Baidu and Alibaba Group Holding, have rushed to upgrade their chat bots so they can handle super long texts of up to 10 million Chinese characters.
The move follows Google, which in February unveiled the latest version of its Gemini large language model (LLM) that achieves a long “context window”, the maximum amount of text an LLM can consider when generating a response, of up to 1 million tokens, or roughly 700,000 English words.
Unlike Google, which only made the update available to “a limited group of developers and enterprise customers” as it is “computationally intensive”, according to its blog, Baidu will in April launch a new version of its Ernie Bot that can process up to 5 million Chinese characters for free, according to a report by Chinese media Chinastarmarket.cn on Friday.
Baidu did not immediately respond to a request for comment on Monday.
Chinese tech giants tout advancing AI models in race to catch up
Similarly, Alibaba said last week that its Tongyi Qianwen chat bot, now able to handle texts comprising around 10 million Chinese characters, would be available for all users free of charge. Alibaba owns the South China Morning Post.
The rush to expand context windows reflects how Chinese AI giants are eager to catch up with Western tech leaders. Baidu and Alibaba were among the first in China to launch their own chat bots, four and five months after ChatGPT, respectively.
The ability to process long texts can help summarise research papers, compare resumes and study computing code, according to Alibaba and Moonshot AI.
But the leap to larger context windows can be constrained by limited computing power. OpenAI’s GPT-4 supports 32,000-token context windows, available for paying subscribers at 6 US cents for every 1,000 tokens as input and 12 US cents for output of the same size.
OpenAI chief executive Sam Altman said in May, before GPT-4 Turbo was unveiled, that expanding access to the 32k model was not a priority for the company as it was “resource constrained”, according to a blog by Joe Reda, chief technology officer at BitBakery Software, who spoke to Altman at an event in Toronto.
In November, OpenAI unveiled a preview version of the more powerful GPT-4 Turbo, which comes with a context window of 128,000 tokens, or about 96,000 English words.
