Nvidia is open-sourcing Audio2Face, its AI-powered tool that generates realistic facial animations for 3D avatars — all based on audio input. The change means developers can now use the tool and its underlying framework to create realistic 3D characters for their games and apps.
Nvidia’s Audio2Face works by analyzing the “acoustic features” of a voice, allowing it to generate animation data that it maps to the 3D avatar’s facial expressions and lip movement. The company notes that developers can use the tool to create 3D characters for pre-scripted content, as well as for livestreams.
Some developers have already used Audio2Face in their games, including Chernobylite 2: Exclusion Zone creator Farm51, as well as the developers of Alien: Rogue Incursion Evolved Edition.
In addition to open sourcing the Audio2Face models and software development kits, Nvidia is making the tool’s training frame available, allowing users to tweak its models for different use cases.
