Google uses AI wizardry to recreate ‘The Wizard of Oz’ in fully immersive 3D – News

“The Wizard of Oz” is being transformed into an immersive 3D experience unlike anything anyone has ever seen, thanks to the use of entirely new artificial intelligence techniques that enhance its resolution and generate additional scenery to fit a gigantic, wraparound LED display.

Sphere Entertainment Co., operator of The Sphere in Las Vegas, said it teamed up with Google LLC and Warner Bros. Studios Inc. to recreate the timeless 1939 classic, transforming it into a “multisensory experience” that promises to immerse viewers in the story like never before.

The reproduction, known as “The Wizard of Oz at Sphere,” will open to audiences on Aug. 28. The producers leveraged some of Google’s most advanced AI models to completely reimagine the classic movie, in a collaboration that marks the beginning of a much broader push into entertainment by the technology giant.

Recreating “The Wizard of Oz”for a 3D environment was not easy. The original flick was shot using a three-strip Technicolor 35-millimeter motion picture camera, which was state-of-the-art at the time but couldn’t be more outdated today. Getting that to fit on Sphere’s 160,000-square-foot curved, 16K resolution display proved to be a monumental challenge, but Google’s AI was apparently more than up to the task.

The 17,600-seat Sphere is an enormous globe structure that’s wrapped in a kind of LED display exoskeleton, making it one of the world’s most unique venues. It has previously hosted immersive concerts by performers such as the Eagles and U2, and has shown a number of movies that were created especially for its immersive screen. But its operators had never tried to adapt an existing film to fit its unusual characteristics.

However, Sphere decided to take on that challenge after landing a licensing deal with Warner Bros, giving it the right to adapt its movie content. The company then turned to Google, asking it to use its AI wizardry to make the magic happen.

Steven Hickson, who is the director for AI foundation research at Google DeepMind, described the immense challenges his team faced in an interview with the Wall Street Journal. “There are scenes where the scarecrow’s nose is like 10 pixels,” he said. “That’s a big technological challenge.”

Google decided that only its most advanced large language models, such as Gemini, Veo 2 and Imagen 3, were up to the task of rendering “The Wizard of Oz” in all of its immersive, 3D glory, but they alone are not enough. In addition, they had to devise new techniques such as “performance generation” and “outpainting” to increase the film’s resolution and extend scenes to include imagery and characters that didn’t exist on the original flick.

That was necessary to make the movie fit with the Sphere’s novel screen, which totally surrounds the viewer to make them appear as if they’re right in the middle of the movie.

Hickson said that in one scene, the movie’s star Dorothy is talking to Aunt Em and Miss Gulch, and it’s known by the viewer that Uncle Henry is in the room too. But in the original, Uncle Henry was not on camera. In the Sphere’s version, viewers will be able to turn around and see Uncle Henry, along with the rest of the house that was not pictured in the original scene. These elements were all generated using AI, Google said.

In a viewing of some scenes at a special preview today, on the even of the Google Cloud Next developer conference in Las Vegas, the results were impressive, though some frames at times seemed to have a slightly artificial appearance. Google and the creators appeared to indicate that it still had a lot of work to do before the late August opening.

Ravi Rajamani, global head of generative AI engineering at Google Cloud, said more than 90% of the movie has been touched by AI. There will also be other AI-generated sensory elements for viewers to enjoy, but Sphere has yet to reveal what they are.

Google partnered with the Los Angeles-based immersive studio company Magnopus LLC on the project, and also consulted with professional Hollywood producers including the Oscar-nominated Jane Rosenthal.

“There was no other company that could do this,” Sphere Chief Executive Jim Dolan said at the preview today that News attended, adding that he had originally approached six companies about the project.

With reporting by Robert Hof

Images: Google