The latest generative models make excellent demos, but are they truly going to transform the way people make movies and television? Not in the near future, according to filmmaking and VFX specialists. However, the long-term changes may be beyond our ability to imagine.
Nikola Todorovic (Wonder Dynamics), Freddy Chavez Olmos (Boxel Studio), and Michael Black (Meshcapade, Max Planck Institute) spoke on a panel at SIGGRAPH in Denver about the potential for generative AI and other systems to modify — but not necessarily enhance — the way media is made today. Their opinion was that, while we can legitimately question the utility of these instruments in the near future, the velocity of invention is such that we should be prepared for significant change at any point thereafter.
One of the first issues addressed was the impracticality of today's video generators.
Todorovic mentioned the "misconception of AI that it's a one-click solution, that it'll get you a final VFX shot, which is actually impossible." Maybe we'll get there, but if you don't have editability, that black box doesn't offer anything. What we're witnessing right now is that the UX is still being explored - these research companies are learning about 3D and filmmaking concepts."
Black argued that language essentially lacks the ability to describe some of the most significant features of visual creativity.
“I mean, things like yoga poses, ballet poses — there’s some classic things we have names for, that we can define, but most of the stuff we do, we don’t have names for,” he said. “And there's a good reason for that: It’s because humans actually have inside them a generative model of behavior. But I don’t have a generative model of images in my head. If I want to explain to you what I’m seeing, I can’t project it out of my eyeballs, and I’m not a good enough artist to draw it for you. So I have to use words, and we have many words to describe the visual world. But if I want to describe to you a particular motion, I don’t have to describe it in words — I just do it for you, and then your motor system sees me and is active in understanding that. And so we, I think it’s a biological reason, a neuro-scientific reason, that we don’t have words for all of our motion.”
That may seem intellectual, but the end result is that text-based prompt systems for imagery have essentially limited control options. Even the hundreds of technical and artistic phrases that are used every day on set and in post-production are inadequate.
Chavez Olmos noted that as a Mexican, he had little opportunities to participate in the cinema field because all of the money and skill was centered in Los Angeles. However, he stated that AI expertise (and demand for it) are more widely dispersed. "I had to leave Mexico because I had no opportunity there; I can see, now, having that same opportunity for people who don't need to go overseas to do it."
However, Black is concerned that quick access to these processes will have unforeseen implications in the short run.
“You can give somebody a powerful car, that doesn’t make them a Formula One driver, right? That’s a little bit like what we have now. People are talking about, everyone’s going to be making films. They’re going to be s—–, quite honestly,” he said. “The democratization thing is exactly what [Chavez Olmos] said, and the power is that maybe some new voice will have an opportunity that they wouldn’t otherwise. But the number of people making really good films is still going to be small, in my opinion.”
"The real revolution," he went on, "the real power of what we're seeing in AI is that we're going to see an entirely new genre of entertainment, and I'm not sure what it will look like." I believe it will be somewhere between video games, films, and real life. The film industry is a form of passive storytelling in which I sit and observe, similar to theater or podcasting. I am the passive recipient of entertainment. But in our daily lives, we tell each other stories, talk about what we did over the weekend, and so on. And that is a highly active form of interactive storytelling.
Before that happens, Chavez Olmos predicts a more traditional adoption curve for AI-generated visuals and performers.
"It's gonna have the same, I think, reaction that we had when we saw the first 'Final Fantasy' movie or 'The Polar Express' — something's not quite there yet, but people are going to start accepting these films," added the comedian. "And, instead of a full CG film, it will be a full AI film, which I believe will be out by the end of this year. I believe people will accept that 'OK, this is AI,' and will move on."
"The crucial thing," Black explained separately, "and Pixar taught us this very clearly: It's all about the story. It's all about connecting with the characters. It's about the heart. And if the film has heart, even if the characters are AI, I believe people will like it," he remarked. "That's not to say they won't want human actors. There's a thrill in knowing it's genuine humans like us, but much better than us; seeing a human at the top of their game encourages all of us, and I don't think that's going to change."