Baidu says latest version of Ernie Bot has surpassed ChatGPT 3.5, and outperforms ChatGPT 4 in Chinese language
- Baidu says tests show Ernie has surpassed GPT 3.5 in comprehensive ability tests and outperformed GPT 4 in Chinese-language abilities
- Baidu CEO Robin Li says Ernie’s ‘cumulative improvement in model performance’ is over 50 per cent since 3.0 was released for beta testing
Baidu has said that the artificial intelligence-powered large language model (LLM) used in its chatbot Ernie Bot, has outperformed OpenAI’s groundbreaking ChatGPT in key metrics, as the race against US and local competitors heats up.
In a statement on Tuesday, the Chinese search and AI giant cited a test by state-run newspaper China Science Daily, which showed that Ernie 3.5, the company’s latest LLM, had surpassed ChatGPT 3.5 in comprehensive ability tests and outperformed the more advanced ChatGPT 4 in several Chinese-language capabilities.
“Just three months after the beta release of Ernie Bot, built on Ernie 3.0, Ernie 3.5 has achieved broad enhancements in efficacy, functionality, and performance,” Haifeng Wang, Baidu’ chief technology officer, said in the statement.
The recent improvements to Ernie’s LLM are already evident in creative writing, questions and answers, reasoning, and code generation, added Wang. The latest Ernie Bot also allows for the expansion of the bot’s features through plug-ins.
The plug-in “Baidu Search” has already been integrated, and equips Ernie Bot with the ability to generate real-time and precise information. The company plans to add more plug-ins from Baidu and third parties.
China Science Daily’s tests were based on benchmarks designed to test AI-language models such as AGIEval, which tests the performance of models via “human-centric standardised exams”, and C-Eval, a Chinese evaluation suite that tests models on thousands of multi-choice questions spanning 52 disciplines.
Li added that training performance had increased twofold, and the inference performance had increased by 17 times, making it faster and cheaper for the company to iterate and upgrade to future versions.
“The Ernie Foundation Model version 3.5 has not only been upgraded on the technical side but also upgraded on the safety side,” said Li, which may reflect moves to address growing content safety concerns from Beijing.
According to Li, generative AI poses a number of governance challenges and healthy development in the space requires the establishment of sound laws and regulations, institutional systems, ethics, and moral guidelines.
The company’s release of Ernie Bot 3.0 in March represented China’s first major response to ChatGPT, prompting companies like Alibaba Group Holding and Tencent Holdings to unveil their own models. Alibaba owns the South China Morning Post.
“It’s foreseeable that large [language] models will permeate into more and more sectors, becoming a key factor driving the digital economy and its integration with the real economy to further grow it,” said Li.
