Adobe has announced its AI-powered Firefly Video Model, a tool designed to enhance video editing workflows in Adobe Premiere Pro and other video tools.
Highlighting the importance of video as a “currency of engagement,” Adobe provided a glimpse into the upcoming Firefly Video Model and the innovative professional workflows it aims to support.
Background
Adobe first launched the Firefly AI in March 2023, rapidly advancing with new models in imaging, design, and vectors.
These models have been integrated into key features across Adobe’s Creative Cloud and Express tools, such as Generative Fill in Photoshop, Generative Remove in Lightroom, Generative Shape Fill in Illustrator, and Text-to-Template in Express.
Since its launch, the Firefly AI has been widely adopted, with users generating over 12 billion images and vectors. Based on feedback from the creative community and enterprise customers, Adobe has developed the Firefly Video Model to advance video editing capabilities.
The new model is designed with creator rights in mind, providing tools to help editors explore creative ideas, fill timeline gaps, and add new elements to footage, all while ensuring content is “commercially safe” and only trained on materials Adobe has permission to use.
Adobe Firefly Video Model Features
The increasing demand for short-form video content has resulted in editors, filmmakers, and content creators needing to do more in less time.
To address these needs, Adobe’s Firefly Video Model incorporates AI to enhance various aspects of video production, from color correction and visual effects to animation and audio mixing.
Key tasks such as removing unwanted objects, smoothing jump cuts, and finding the perfect b-roll can now be completed more efficiently, allowing editors to focus on creating a compelling narrative.
Adobe’s AI tools also streamline collaboration, integrating with tools like Frame.io for a smooth review and approval process.
Firefly Text-to-Video
The Firefly Video Model includes a “Text-to-Video” feature, allowing users to generate B-roll and other footage using text prompts, camera controls, and reference images. Key capabilities of this model include:
- Generating videos of the natural world, such as landscapes, plants, or animals, in under two minutes.
- Creating complementary shots using reference frames and rich camera controls (e.g., angle, motion, zoom).
- Supporting various use cases like atmospheric elements (e.g., fire, smoke, water) that can be layered over existing content.
- Ideating 2D and 3D animations, including Claymation.
- Brainstorming custom text effects.
- Transforming existing still shots or illustrations into live-action clips through the “Image-to-Video” feature.
Additionally, a feature called “Generative Extend” will be available in Premiere Pro (beta) later this year. This will allow users to extend clips, smooth transitions, or hold shots longer for precise edits.
Availability
The Adobe Firefly Video Model will enter beta later this year. Interested users can sign up for the waitlist on Adobe’s official website.