The company turning TikTok moves into avatars, without a depth sensor
YunBo’s tech can already be found in games from Tencent and Perfect World, but it’s bringing motion capture to the masses with a new mini program on WeChat and Douyin
Imagine if you could customise your Fortnite avatar to bust out the exact same dance moves you put on TikTok with just a click of a button. For some Chinese games, this is already happening thanks to an AI company called YunBo.
Now YunBo says it has an algorithm that doesn’t need a depth sensor, and it will soon bring its new technology to the masses with a new mini program.
The mini program, called XiaoK Show, will launch on WeChat and Douyin, China’s version of TikTok, by the end of March. But I recently got to give the demo a try. The app lets users record themselves dancing and turns it into a virtual character showing off all the same moves within minutes.
XiaoK Show appears to be a simple utility app, but it does what it promises. To get started, you upload a short dancing clip and select the cartoon character you want to mimic your moves. You can then download the video and share it on other social media platforms.
The demo wasn’t especially polished and the number of characters and backgrounds available was limited. But the results are impressively accurate.
The most impressive part of XiaoK Show, though, is that it can run on any phone with WeChat or Douyin. And that’s just about every smartphone in China these days. Without the need for a depth sensor, people don’t need to buy a US$1,000 iPhone to try out this motion capture tech.
“Our algorithm is smart enough that you don’t even need a depth sensor anymore,” YunBo co-founder Mei Song said. “It can just simulate depth on its own.”
The algorithm is also able to generate depth data that can be stored in a file. Mei said extrapolating depth data from any recording is what sets YunBo’s algorithm apart. This means anyone can grab a video online and apply it to their own avatar, he added.
Before starting YunBo three years ago, Mei and his co-founder Chen Min worked at Chinese gaming heavyweight Linekong Interactive as senior VP and CTO respectively. But after 10 years at the company, the two left to start their own business so they could focus on creating technology instead of designing games.
And that’s what they’re doing now with XiaoK Show, which Mei said he hopes will “turn everyone into an animator”. A full-fledged app is also in the works, Mei said.
Mei acknowledges that the motion capture business is bound to be competitive going forward. But he believes that his company, which currently has just a dozen employees, is in a good position to make motion capture popular among average smartphone users.
“I wouldn’t say that nobody can do what we do,” Mei confessed. “But we definitely got in early and are ahead of the pack.”
Purchase the China AI Report 2020 brought to you by SCMP Research and enjoy a 20% discount (original price US$400). This 60-page all new intelligence report gives you first-hand insights and analysis into the latest industry developments and intelligence about China AI. Get exclusive access to our webinars for continuous learning, and interact with China AI executives in live Q&A. Offer valid until 31 March 2020.
