Waymo Is Using Google's Genie 3 AI To Practice Handling Tornadoes, Elephants

Waymo is looking to improve how its self-driving vehicles react when faced with unique scenarios, and it’s leveraging Google’s new Genie 3 world engine AI model to do it. It’s testing everything from sudden tornadoes and heavy snow conditions to deep flood waters and wild animal encounters.

When Google launched its Genie 3 world engine for Gemini Ultra Plan subscribers last week, it was quickly picked up as a tool to shortcut game prototype development. But one of the more realistic ways to use world engines is in training robots and other physical AI devices to understand the world around them. So, Waymo (a subsidiary of Google parent company Alphabet) is using this fast world-building system to test out its driving AI in the kind of scenarios that are hard to plan for in the real world.

“By simulating the ‘impossible,’ we proactively prepare the Waymo Driver for some of the most rare and complex scenarios,” Waymo said in a blog post. “This creates a more rigorous safety benchmark, ensuring the Waymo Driver can navigate long-tail challenges long before it encounters them in the real world.”

The Waymo World Model is based on Genie 3, but designed to be more realistic than the Zelda clones people have been producing. Waymo uses it to create interactive driving environments, then converts the 2D video output into 3D LiDAR that can be applied to Waymo’s hardware.

Waymo even figured out how get around Genie 3’s limited long-term stability. The model tends to fall down and lose its consistency and object permanence after a minute or so, but Waymo found that by speeding up the footage derived from it by 4x, it could create much longer scenarios. They’d be horrible to play as a game, but are invaluable for an AI training simulation.

Recommended by Our Editors

Waymo had its robotaxis encounter animals like elephants, car accidents, wildfires, snowstorms, and heavy flooding. It also trialed different scenes, like heavy traffic and busy crosswalks, a semi blocking the road, and even an enormous tumbleweed the size of a car.

While some of these situations are ridiculous, and many of them extremely unlikely to ever occur, they give Waymo an injection of additional data that’s close enough to the real thing that its robotaxis should be better equipped should they ever encounter these odd situations.