In December, Meta announced a multiyear partnership with James Cameron’s Lightstorm Vision to bring 3D entertainment to Meta’s Quest headsets. This week, Cameron joined Meta CTO Andrew Bosworth on stage during the company’s Meta Connect conference to share a first result of that partnership: Quest owners are able to watch an exclusive preview clip for Cameron’s upcoming Avatar 3 movie via the headset’s new Horizon TV app.
I had a chance to sit down with the duo ahead of their keynote appearance to talk about the potential that mixed reality headsets represent for 3D video, the complicated history of 3D TV, and Cameron’s attitude toward generative AI.
This interview has been edited for length and conciseness.
Why did you partner with Meta on 3D entertainment?
James Cameron: It just seems like a natural convergence. I had been proselytizing about stereoscopic media and entertainment for 25 years. It kind of went dormant for a while, because cinema was the only place to really see it. It had a brief life on flat panel TV, but the devices never really worked that well.
But in mixed reality headsets, you’re innately a stereoscopic viewer. [When I saw] mixed reality headsets, it occurred to me: It’s time to bring back the capability that I had spent 15, 16 years developing. So I stood up a new company, Lightstorm Vision, around stereoscopic production. At the same time, the Meta content team was looking for a partner in stereoscopic production [to] break into entertainment, meet the studios and filmmakers.
Andrew Bosworth: [We] were looking for each other without realizing it. We kind of pitched each other. It continues to be a tremendous partnership for us. [James isn’t just a great storyteller,] but also an innovator who can tell us: Here’s the ways in which the thing that you’re building is not meeting the needs of storytellers. To have somebody who is both an expert and a critic is an extremely high value for us.
A lot of VR storytelling has focused on putting viewers into the middle of the movie, making things more interactive. It seems like you’re more about 3D stereoscopic, framed lean-back entertainment?
Cameron: You’re right. What I’ve spent a career doing is telling stories in a linear narrative format. Sometimes, those are documentary stories. Sometimes, those are completely fictional stories. But it’s in a rectangle.
Everybody was quick to discount the rectangle. But what that rectangle does is it directs the eye. Avatar movies are an example of what I like to do, which is give you a lot of things to enjoy within a frame. But the frame is the frame. The frame is telling the story.
We have this hundred-plus year cinematic vocabulary [that] maps to the way the mind writes memory. That’s why cinematic vocabulary is identical in China, India, Japan, the Americas, Europe. We all think through that rectangular window the same way, because it’s how the brain works.
Bosworth: The timing of this is [key.] Why didn’t we embrace this hundred-year vocabulary earlier? Partly, we just didn’t have the displays for it. You had a TV. You had a phone. Why would you watch a movie in the headset when the resolution was not as good?
What’s different now is that we have the resolution. We have brightness actually in excess of what people are seeing in these other environments. We have a refresh rate that’s really high. We’re at the point with the headsets that we have in the market today, not just ours, where we can do what the TV does, and also immersive media. I think there is probably room for both.
Cameron: I think episodic television is the big overlooked thing here. Stereoscopic production enhances your sense of engagement with the people you’re seeing in the frame. But there hasn’t been a way to distribute episodic television stereoscopically to date. That’s changing, and I think it’s going to be huge.
It’s interesting you mentioned 3D TVs. Those obviously flopped, and sometimes people say: VR is going to be the next 3D TV. Now you’re telling me: Yes, VR is going to be the next 3D TV, but it’s actually going to work.
Bosworth: A funny thing happens where people say: Oh, 3D is all the same. The first time I went to visit James in L.A., he showed us a piece of his upcoming film. He showed it to us how the average stereoscopic movie theater projects it, which is relatively low brightness. Then he put it on a laser projector, which is how he intended it to be seen in theaters. It’s a profound difference.
One of the problems that you do have is that audiences think they’re all the same. You have to be relatively savvy to understand what Dolby Vision means, or what IMAX means. 3D TV was a relatively poor stereoscopic experience. Some of it was the glasses. Some of it was limited depth projection. Some of it was brightness …
Bosworth: Totally. It wasn’t a great experience. The nice thing about the headset is you’re guaranteed an outstanding experience every single time. It’s a different product, and I think to some degree, we do ourselves a disservice when we in the consumer electronic space overly flatten things.
James, you’re on the board of Stability AI. What gets you as a filmmaker, and as a CGI pioneer, excited about generative AI?
Cameron: I look at the cost of VFX these days as becoming quite limiting for the types of movies and shows that get greenlit. The labor rates have gone up significantly, and the theatrical market has partially collapsed, at least 30 percent. And so we need a solution.
The solution will probably lie in creating specific custom gen AI models that can be injected into existing visual effects workflows. I’m less interested in a kind of magic wand text-to-video approach. If I were a young filmmaker with no resources, no money, and couldn’t afford actors, I would be very interested in that type of production. But that’s not my interest.
My interest is [in] mainstream and high-end production that involves a lot of effects. And I’m not anti-artist at all. I don’t want to cut people. I don’t want people to lose their jobs. What I want them to do is be more productive, so that we can have more throughput through the existing companies.
Through Stability, I [have met] a lot of gen AI developers. Great people, but they’re making stuff in a vacuum. They’ve never made a shot for a movie from end to end. All the production-focused tools that were built over the last 30-some years in CG and in VFX were created because productions needed them. They weren’t created in a vacuum.
Do you think generative AI is going to democratize filmmaking for that young filmmaker though? People used to say the same thing about game engines, real-time and virtual production tools, and that’s not exactly what happened. Instead, it has led to Hollywood using gigantic, hugely expensive LED screens …
Cameron: I think it’s going to create entry-level avenues for people who can come [to Hollywood] with a film that they’ve made using prompts. I think it’s going to make it easier to get into [that] system, but I don’t think the system will change fundamentally.
I personally hope we never replace actors. To me, the joy of the process is working with other artists, creating a moment, an authentic moment, an emotional moment, creating characters.
People say: Gen AI can’t be as creative as humans. I think that’s dead wrong. I think it could be just as creative. What it can’t do is create that unique lived experience of an individual viewpoint, which is what we love the most in literature, in novels and film. It can’t do that, but it can be in service of that unique vision. And I intend to embrace it as much as I can, but always in service of the creative process.
This is Lowpass by Janko Roettgers, a column on the ever-evolving intersection of tech and entertainment, syndicated just for The Verge subscribers once a week.
0 Comments
