Meta is bringing its AI video tricks to Instagram, with new features that’ll let users modify their videos just by using text to modify imagery—from adding bling to changing backgrounds, and doing pretty much anything you can imagine.
“I’m super excited about Movie Gen, our early AI research model that will let you change nearly any aspect of your videos with a simple text prompt,” Instagram head Adam Mosseri said in a video announcement.
In the video, Mosseri appears in his original clothes and location—a grey jacket and a normal-looking bedroom—and the scenery starts to randomly change, showing him in Paris, New York, and a pool, and switching his clothes to different things like a leather jacket or a puffy sweater while in the Himalayas.
“You should be able to change your outfit or change the context in which you’re sitting, or add a chain, whatever you can think of,” Mosseri explained, showing off demos where he seamlessly switched backgrounds and transformed his appearance.
Meta announced Movie Gen a few months ago, showcasing some generations that exceeded other models’ outputs in quality and prompt adherence. The company also showed some AI-generated Instagram reels created by social media influencers like Paige Piskin, and social media brand 10pm Curfew’s Girls and K5sh accounts.
Beyond generative video, Movie Gen handles video inpainting—altering specific things inside of a video while leaving all the other elements intact. This makes the model extremely versatile, and gives users the capability to do a lot of different visual manipulations—from changing a person’s outfit to swapping out backgrounds, and even transforming people into puppet-like cartoon versions of themselves by manipulating the overall style of the scene
The technology appears to build on Meta’s Movie Gen AI model, which Decrypt reported on last year. It’s the company’s answer to similar tools from rivals, like OpenAI’s recently announced Sora video generator, Runway’s inpainting feature, and even some open-source tools like Propainter.
For Instagram, this marks the first time Meta is bringing its video AI capabilities directly to users. Movie Gen can create entirely new 16-second videos or modify existing ones, complete with synchronized sound effects and music.
The generative video scene has been on a run lately, with very important developments announced in the last months after a somewhat quiet year with spaced-out releases.
Sora is finally out, but there have been a lot of interesting alternatives—like Invideo V3 (capable of generating entire scripts, scenes, and 10-minute-long videos out of a single idea), Dream Machine (which can generate coherent and realistic videos even in surreal scenarios), Google’s Veo2 (capable of generating highly realistic videos with great prompt adherence), and Showrunner (which is able to generate small cartoon sketches).
Meta hasn’t announced a specific launch date, but Mosseri said it’s coming in 2024.
Edited by Andrew Hayward
Generally Intelligent Newsletter
A weekly AI journey narrated by Gen, a generative AI model.
