“Salt” is the brainchild of Fabian Stelzer. He’s not a filmmaker, but for the past few months he’s been relying largely on artificial intelligence tools to create this series of short films, which he releases on Twitter roughly every few weeks.
Stelzer creates images using image generation tools such as Stable Diffusion, Midjourney, and DALL-E 2. He usually creates voices using AI speech generation tools such as Synthesia or Murf. And he uses GPT-3, a text generator, to help write the script.
There is also an element of audience participation. After each new episode, viewers can vote on what should happen. Stelzer takes the results of these polls and incorporates them into the plot of future movies, which he can run faster than a traditional filmmaker as he uses these AI tools.
“In my small home office studio, I can make a 70s science fiction movie if I want to,” Stelzer, who lives in Berlin, said in an interview with CNN Business from that studio. “And actually I can do more than a sci-fi movie. I can think about, ‘What is the movie in this paradigm, where the execution is as simple as an idea?'”
The plot is, at least for now, still vague. As the trailer shows, it generally focuses on a distant planet, Kaplan 3, where an overabundance of what initially appears to be mineral salt leads to dangerous situations, such as endangering it in some way. of an approaching spaceship. To make things even more confusing (and intriguing), several storylines have also been introduced and maybe even some temporary anomalies.The resulting films are beautiful, mysterious and foreboding. So far, each movie is less than two minutes long, consistent with Twitter’s maximum video length of two minutes and 20 seconds. Occasionally, Stelzer will tweet a still image and caption that add to the series’ strange, alien mythology. Just as AI image generators have already made some artists nervous, Stelzer’s experiment provides an early example of how disruptive AI systems could be for movie making. As AI tools that can produce images, text and voices become more powerful and accessible, this could change the way we think about idea generation and execution – challenging what it means to create and be a creator. While the following is limited for these videos, some in the tech space are watching closely and expect more to follow.
“Right now it’s in an embryonic stage, but I have a whole bunch of ideas about where I want to take this,” Stelzer said.
“Shades of Ideas and Story Seeds”
The idea for “Salt” grew out of Stelzer’s experiments with Midjourney, a powerful, publicly available AI system that allows users to enter a text prompt and get an image in response. The prompts he fed the system generated images that he said “felt like a movie world,” depicting alien vegetation, a mysterious figure lurking in the shadows, and a strange-looking research station on a barren mining planet. One image contained what appeared to be salt crystals, he said.
“I saw this and thought, ‘Okay, I don’t know what’s happening in this world, but I know there are a lot of stories, interesting things,'” he said. “I saw narrative shades and shadows of ideas and story seeds.”
Stelzer has a background in AI: he co-founded a company called EyeQuant in 2009 which was sold in 2018. But he doesn’t know much about making movies, so he started teaching himself software and made a “Salt” trailer, which he tweeted on June 14 with no written introduction. (The tweet did, however, contain a salt shaker emoji.) That was followed a few days later by what Stelzer calls the first episode. He has released several so far, along with numerous still images and some short film clips. Ultimately, he hopes to cut the pieces of “Salt” into one feature film, he said, and build a related company to make films with AI. He said it takes about half a day to make each film.
The vintage sci-fi vibe is partly a tribute to a genre Stelzer loves and partly a necessity due to the technical limits of AI image generators, which are still no good at producing images with hi-fi textures. To get AI to generate the images, he creates prompts with phrases like “a sci-fi research post near a mining cave,” “35mm footage,” “dark and beige atmosphere,” and “salt crusts on the wall.”
The appearance of the film also fits Stelzer’s editing style as an amateur author. Using AI to generate still images for “Salt”, Stelzer uses some simple techniques to make the scenes look animated, such as shaking parts of an image to make it look like they’re moving or zooming in and out. It’s gross, but effective.
“Salt” goes to college
“Salt” has a small but charmed following online. The Twitter account for the film series had about 4,500 followers on Wednesday. Some of them have asked Stelzer to show how he makes his films, he said.
Savannah Niles, director of product and design at Magnopus, the maker of AR and VR experiences, follows “Salt” on Twitter and said she sees it as a prototype of the future of storytelling — when people actively participate and contribute to a story that helps build AI. She hopes that tools like those Stelzer can eventually make it cheaper and faster to produce movies, which today can involve hundreds of people, take several years, and cost millions of dollars.
“I think a lot of these will come, which is exciting,” she said.
It is also used as a teaching aid. David Gunkel, a professor at Northern Illinois University who has watched the movies via Twitter, said he previously used a short sci-fi movie called “Sunspring” to teach his students about computational creativity. Released in 2016 and starring ‘Silicon Valley’ actor Thomas Middleditch, it is believed to be the first film to use AI to write its script. Now he plans to use “Salt” in his communications technology classes in the fall, he said.
“It creates a world that you feel involved, immersed in,” he said. “I just want to see more of what’s possible and what will come out of this.”
Stelzer said he has a “somewhat cohesive” idea of what the overall story structure of “Salt” will be, but he’s not sure he wants to reveal it — in part because the community involvement has already done the story in a way. deviate from what he intended.
“I’m actually not sure if the story I have in my head is going to end like this,” he said. “And the charm of the experiment for me, intellectually, is driven by the curiosity to see what I as a creator and the community can come up with together.”