The latest generative models are great for demos, but will they really change the way people make movies and TV? Not in the short term, according to film and VFX experts. But in the long term, the changes could be literally beyond our imagination.
At a panel discussion at SIGGRAPH in Denver, Nikola Todorovic (Wonder Dynamics), Freddy Chavez Olmos (Boxel Studio) and Michael Black (Meshcapade, Max Planck Institute) discussed the potential of generative AI and other systems to change – but not necessarily improve – the way media is produced. They agreed that while we can rightly question the usefulness of these tools in the immediate future, the speed of innovation is so high that we must be prepared for radical change at any time in the future.
One of the first topics discussed was the impracticality of today’s video generators.
Todorovic noted the “misconception of AI that it’s a one-click solution that gives you a final VFX shot, and that’s really impossible. Maybe we’re getting there, but if you don’t have editability, that black box doesn’t do you much good. What we’re seeing right now is that the UX is still being discovered – these research companies are starting to learn the methods of 3D and the terms of filmmaking.”
Black pointed out that language fundamentally lacks the ability to describe some of the most important aspects of visual design.
“I mean, things like yoga poses, ballet poses – there are some classic things that we have names for, that we can define, but for most of the things we do, we don’t have names,” he said. “And there’s a good reason for that: Humans actually have a generative behavior model within them. But I don’t have a generative model for Pictures in my head. If I want to explain to you what I see, I can’t project it out of my eyeballs, and I’m not a good enough artist to draw it for you. So I have to use words, and we have a lot of words to describe the visual world. But if I want to describe a particular movement to you, I don’t have to describe it in words – I just do it for you, and then your motor system sees me and is actively trying to understand that. And I think there’s a biological reason, a neuroscientific reason, that we don’t have words for all of our movements.”
This may sound a little philosophical, but the upshot is that text-based prompting systems for images are fundamentally limited in their controllability. Even the hundreds of technical and artistic terms used daily on set and in post-production are inadequate.
Chavez Olmos pointed out that as a Mexican, he had few opportunities to get involved in the film industry because all the money and expertise was concentrated in LA. However, he said AI expertise (and demand for it) is more widespread. “I had to leave Mexico because I didn’t have a chance there. I imagine that the same opportunities are now available for people who don’t have to go abroad.”
However, Black fears that sudden access to these processes could have unintended consequences in the short term.
“You can give someone a powerful car, but that doesn’t make them a Formula 1 driver, does it? It’s a bit like what we have now. People talk about everyone making films. Quite frankly, they’re going to suck,” he said. “The democratization thing is exactly what (Chavez Olmos) said, and the power is that maybe a new voice will get a chance that it wouldn’t otherwise have. But the number of people making really good films will still be small, in my opinion.”
“The real revolution,” he continued, “the real power of what we’re seeing in AI is that we’re going to see a whole new genre of entertainment, and I don’t know exactly what it’s going to look like. I predict it’s going to be something between video games, movies and real life. The film industry is passive storytelling: I sit there and watch, it’s like the theater or a podcast. I’m the passive recipient of the entertainment. But in our daily lives, we tell each other stories, we talk about what we did at the weekend and so on. And that’s a very active kind of interactive storytelling.”
Before that happens, however, Chavez Olmos expects a more traditional adoption curve for AI-generated images and actors.
“I think it will have the same reaction as the first Final Fantasy movie or The Polar Express – it won’t be quite there yet, but people will start to accept those movies,” he said. “And instead of a pure computer animation movie, it will be a pure AI movie, which I think we’ll see even by the end of this year. I think people will get over that and think, ‘OK, this is AI,’ people will accept it.”
“The most important thing,” Black said separately, “and Pixar taught us this very clearly: It’s about the story. It’s about building a connection with the characters. It’s about heart. And if the movie has heart, it doesn’t matter if the characters are AI, I think people will enjoy the movie,” he said. “That doesn’t mean they don’t want human actors. It’s exciting to know that they’re real people like us, but way better than us. Seeing a human at the peak of their powers inspires all of us, and I don’t think that’s going to go away.”
