Nvidia has announced the Omniverse Avatar Cloud Engine for Games, integrating several disparate AI-based features and functionalities into one comprehensive gaming AI package. AI-based upgrades for video game NPCs have long been touted as one of the most interesting use cases for modern-day AIs, and it would appear that Nvidia may be making one of the first serious forays into this endeavor.
The thing that sets Nvidia apart from its competitors is that the company is going all-in on AI. For years, Nvidia's Deep Learning Super Sampling has been at the forefront of game upscaling technologies, thanks to its Tensor cores and how they are leveraged and expounded upon in subsequent GPU generations, which has worked out to a fairly significant edge in ray tracing performance, for example.
Though ray tracing is an important aspect of rendering, Nvidia is ready to take things to the next level with further AI integrations, it would seem. Namely, the Omniverse Avatar Cloud Engine, or ACE, for Games technology allows NPCs with no pre-written dialogue to communicate with the player in real-time, with proper voiced lines. The kicker is that ACE for Games is, according to Nvidia CEO Jensen Huang, "basically a large language model" that's been reappropriated for video game integration. ACE for Games combines the understanding of natural language, text-to-speech, and facial animation to have NPCs listen to and interact with players in real time.
Though Nvidia's GPU business in 2023 has been critiqued for the disappointing performance of some of its RTX 4000 GPUs, the company is still firing on all cylinders on other fronts. ACE could, in theory, revolutionize AI interaction in games, but there are still big questions that Nvidia will need to answer. The obvious issue, for example, is that of running the AI integrations in real time; Nvidia hasn't been clear on exactly what it takes for ACE for Games to run properly.
Of course, the odds are good that ACE for Games will be very interesting to see in action once it finally picks up steam. For the time being, the company's most impressive real-world AI use case is still ray tracing. One of the most prominent and impressive examples of ray tracing is that of Cyberpunk 2077's RT Overdrive mode, which has come up due to Nvidia's cooperation with developer CD Projekt RED.
Nvidia's AI-based tech has proven to be very flexible in the past, with modders able to integrate it into games that don't actually support such features by default. A prime example is that of a modder setting up Nvidia DLSS in Fallout 4, alongside AMD and Intel's own upscaling technologies. It's possible that ACE for Games will be similarly flexible, but only time will tell.
Source: Nvidia