Why the secret AI sauce behind TikTok is such a vital ingredient in luring potential buyers
- ByteDance has long been a proponent of content recommendation systems, and uses it on other products, such as the popular news aggregator Jinri Toutiao
When ByteDance rebranded the US teen karaoke app Musical.ly it acquired as TikTok in 2018, it was just another short video app for American teens.
Today it is the most downloaded app in the world, proving so popular that it has become a flashpoint in the escalating US-China tech war, which was previously focused on heavy-duty areas such as chips and 5G networks.
Among the newly-added export restrictions – the first time China has updated such rules in over a decade – are “personalised information push technologies based on data analysis” and “artificial intelligence interactive interfaces”.
Both of these tools are used to build ByteDance’s powerful recommendation system, which feeds curated content to users based on their interests and activity. While Musical.ly may have given ByteDance a foothold in the US market, it was the secret sauce – its algorithm – that allowed it to build momentum.
China reiterates right to review tech export deals amid confused TikTok US sale
At the same time, the app also feeds in a certain amount of video content outside the user’s direct interests.
But why is the algorithm so important?
TikTok was the most-downloaded non-game app in the world in the first half of 2020, attracting more than 596 million installs, excluding its Chinese version Douyin, according to analytics firm Sensor Tower.
Although the basics of the algorithm TikTok uses are similar to ones found in apps from other tech companies, it is the special features that each company can add that differentiates the AI engine, said Wong Kam-fai, a professor in engineering at the Chinese University of Hong Kong and one of the first batch of national experts appointed by the Chinese Association for Artificial Intelligence.
Wong, who does not believe TikTok’s AI engine is truly unique, said a new recommendation system could be built for the short video app with fresh user data, in one year‘s time or so, but losing the existing tool would have “a very big impact” on TikTok’s current valuation.
“The technology works only when the algorithm and user data are both good. Part of the reason why ByteDance’s apps have an advantage over the competition are their user data,” said Hao Peiqiang, widely known as ‘Tinyfool’ who worked as a software engineer and now runs a tech blog and advises companies.
“The regulation [on privacy] in China is too weak and the awareness of privacy protection is relatively low,” said Hao, referring to the treasure trove of user data ByteDance has been able to amass via the app.
TikTok has repeatedly said that it stores US user data outside China and that the data is not subject to Chinese law.
Wong pointed out that some users, and in turn investors, may not be prepared to wait for the time it takes to build a new algorithm.
China’s new export rules mean quick TikTok sale ‘unlikely’
“You can’t wait for the team to rebuild the algorithm as TikTok is already very popular,” said Wong. “It’s like your favourite TV show closing because of a technical issue … I don’t think users will accept that.”
“For bidders such as Microsoft and Walmart, they want to buy the app and make it work immediately,” said Wong. “But if they need to wait a while to make it work well, maybe they won’t want to buy it any more.”
Not all experts agree that TikTok’s AI engine is truly unique though.
“While Tiktok could not exist as-is without its recommender system, that doesn’t exactly mean the system is anything special,” said Julian McAuley, an associate professor at the University of California San Diego, who studies the field.
“The early drivers of recommender systems included e-commerce companies. For example, Amazon has used recommendation technology for almost two decades, though early systems involved simple item-to-item similarity, rather than anything machine learning-based,” said UCSD’s McAuley.
“Netflix was also a big driver of recommendation technology in the mid-2000s, culminating in things like the Netflix Prize (2006), which led to a lot of academic interest in recommendation technology,” said McAuley.
US will target more Chinese apps after TikTok, WeChat, Trump adviser says
In the modern smartphone era, the technology has however been criticised for the so-called “filter bubble” problem, whereby users surround themselves with content that serves to reinforce their own biases – rejecting all information that does not conform with their own world view.
“Companies want to optimise engagement metrics and don’t want to inject diverse or more balanced content if doing so hurts their key metrics,” said McAuley, adding they have little incentive to solve the problem.
“We’re living in a time when our demand for preferred information has never been so high.”