South China Morning Post
Advertisement
Advertisement
NetEase
Get more with myNEWS
A personalised news feed of stories that matter to you
Learn more
NetEase’s AI can use a standard portrait picture to render a 3D game character. (Picture: NetEase)
AbacusGames

NetEase can use AI to turn you into a game avatar with one selfie

NetEase is investing heavily in AI to create better gaming experiences, but it has to contend with rivals like Tencent and Reallusion

NetEase
Josh Ye
Josh Ye
Why you can trust SCMP
This article originally appeared on ABACUS

Have you ever wanted to play as yourself in a video game? NetEase has given this old dream an upgrade with artificial intelligence, allowing the company’s flagship MMORPG game, Justice, to construct an in-game model of a player from a single selfie.

A paper from NetEase’s Fuxi Lab, which works on AI, shows how NetEase can reconstruct a person’s image into a 3D avatar from a 2D image. Once the avatar is built, it can start dancing to AI-generated music or be used to talk to AI in-game assistants who compose their own poems.

NetEase said the feature has already been employed by Chinese gamers more than 1 million times. It's a step up from most character creation systems currently available in games, which require players to manually put in hundreds of parameters before getting a lifelike character.

“[Going forward],0 2D pictures will create not only faces but also hairs, expressions and even movements that can make the entire character more lifelike,” NetEase told us.

“Now this technology is just a feature in the game. But we will let game developers create game characters with this technology in the future,” they added.

But these AI-generated avatars don’t just spring fully-formed from a single image. It does require a little user input. A player first has to adjust a few sliders to get the avatar looking just right.

Fuxi Lab prides itself on its ability to create lifelike motion using AI. (Picture: NetEase)

AI helps NetEase take this tech further. The company trains its system to understand the bone features in human faces. Current face reconstruction schemes available on the market treat skin like molded mesh, according to NetEase. But by understanding facial bone structure, this new system can quickly render 3D game characters with minimal user input.

The effect is an avatar that looks like it actually belongs in the game. This is perhaps the most challenging part of creating avatars based on real people.

An avatar has to look and feel consistent with the rest of the virtual world. Simply projecting an extremely detailed human face onto an animated character results in what’s called the uncanny valley. The eerie effect of seeing something that doesn’t look natural could be off-putting to players. But thanks to AI, NetEase can now recreate a face that fits with the rest of a game’s aesthetics.

NetEase has been investing a lot in AI. In addition to enabling better in-game effects, the company hopes it will lead to better interactive experiences for gamers and help game developers.

The company has multiple divisions working on AI, but its most prominent is Fuxi Lab. Fuxi brands itself as China’s first lab focusing on developing AI in video games. The group has also been invited to speak at the marquee panel of the upcoming Game Developers Conference in California, making Fuxi Lab the first Chinese research lab to make an appearance at the main event of the world’s biggest game developer conference. 
As NetEase continues to invest in AI, it could benefit the company in a number of ways, according to Chengdu Gaming Federation founder Charlie Moseley.

“I think Netease (and China) understands that perpetual AI research and development will expand their toolset and capabilities in the game,” he said. He added that potential benefits range “from developing new gameplay mechanics, to populating game content, to potentially even finding new business models.”

But NetEase isn’t alone in using AI to make more realistic characters in games. Tencent, its biggest domestic rival, is also using AI and photogrammetry in a system called “ Any Face” to create life-like facial animations.
US-based Reallusion is another leader in this area. Its Headshot character creation tool, which can also create lifelike game avatars, is being used by industry professionals like Zeng Xiancheng, creator of popular indie game Bright Memory. Zeng said that that as a developer, Reallusion’s Headshot technology is just as effective as the algorithm devised by NetEase.

“This technology developed by NetEase is certainly very strong,” Zeng said. “[But] China as a whole is still just beginning to see AI in video games. Many studios still have to use software made by foreign companies.”

For more insights into China tech, sign up for our tech newsletters, subscribe to our award-winning Inside China Tech podcast, and download the comprehensive 2019 China Internet Report. Also roam China Tech City, an award-winning interactive digital map at our sister site Abacus.

Post