Baidu tops Microsoft and Google in teaching AI to understand human language, thanks to differences between Chinese and English

Chinese search giant Baidu has beaten Microsoft and Google in an ongoing natural language processing competition, thanks to the linguistic differences between Chinese and English.

Baidu’s model, called ERNIE (Enhanced Representation through kNowledge IntEgration), recorded the highest score of 90.1 – just ahead of Microsoft and Google – on the General Language Understanding Evaluation (GLUE) benchmark and analysis platform for natural language understanding. Baidu’s model was first developed to understand Chinese language but researchers soon found it was able to understand English better too.

Baidu’s ERNIE was inspired by Google’s BERT, a “masked” language model used by the US tech giant to train AI to understand human language. Google’s model hides 15 per cent of the words in each sequence and then tries to predict them based on the context.

However, given that many Chinese characters do not have an inherent meaning until they are strung together with other characters – a key linguistic difference from English – the team at Baidu needed to train its AI model to understand how to hide a string of meaningful characters and predict the masked ones.

The team at Baidu illustrated the technique on its Github page, taking “Harry Potter is a series of fantasy novels written by J. K. Rowling”, as an example.

BERT was able to identify the “K” through the local co-occurring words J, K, and Rowling, but was not able to learn any knowledge related to the word “J. K. Rowling”. However, ERNIE was able to extrapolate the relationship between Harry Potter and J. K. Rowling by analysing implicit knowledge of words and entities, to infer that Harry Potter was a novel written by J. K. Rowling.