Last summer, Baidu CEO Robin Li claimed the company’s flagship large language model (LLM) Ernie surpassed OpenAI’s Chat-GPT on multiple fronts, including Chinese language capabilities. Now, on the eve of Ernie’s first anniversary, Li is highlighting another competitive advantage of the product – writing classical Chinese poetry.
In an interview with Chinese state television last week, Li revealed that Ernie performed better than ChatGPT when asked to generate a poem in Qinyuanchen, a particularly complex poetry meter dating back to the Tang Dynasty.
“If I asked GPT to compose a poem following the Qinyuanchun scheme, the tool would become totally confused,” Li said, “because it lacks the understanding of whether the first sentence should consist of four words or five.” In the interview, Li also acknowledged that Ernie’s English ability still lags behind GPT’s, rowing back from his remark in October that Ernie is “not inferior in any aspect” to GPT-4, the most recent iteration of ChatGPT.
Since its launch in November 2022, ChatGPT has become the benchmark to which all other generative AI chatbots must aspire. Ernie fares well in this regard, ranking top among Chinese alternatives according to the domestic think tank Xinhua Institute, which tested LLMs’ basic language skills, logical reasoning, subject knowledge, and ability to enhance professionals’ productivity.
While Ernie was trained in a Chinese context supported by Baidu’s massive data resources inside the walled garden of China’s internet ecosystem, ChatGPT is tailored to the English-speaking world. Much as GPT cannot accurately generate a Qinyuanchun poem, Ernie struggled when prompted to write a haiku. Interestingly, Ernie possesses up-to-date knowledge of current affairs, the American model’s Achilles heel.
Ernie, which became open to the Chinese public in August 2023, remains unavailable outside China.
