Adobe is working on an AI model to create video content. The exact release date and details about the model are yet to be disclosed by the company.
As a response to competitors like OpenAI’s Sora and Google’s Imagen 2, Adobe is adding a generative AI model to its Firefly line of products. This new tool will become part of Adobe’s Premiere Pro, expected to be integrated later this year.
Similar to other generative AI video technologies available, Adobe’s model generates video content either from a prompt or reference images and introduces three new features in Premiere Pro: object addition, object removal, and generative extend.
These features are quite self-explanatory.
Object addition enables users to select a portion of a video clip and add objects to it based on a prompt. In a demonstration, Adobe showcased a generated image of a briefcase filled with diamonds using their model.
Image Credits: AI-generated diamonds, courtesy of Adobe.
Object removal allows for the elimination of unwanted objects from video clips, such as microphones or background distractions like coffee cups.
Removing objects with AI. Note that the results may not be flawless. Image Credits: Adobe
Generative extend functionally adds extra frames at the start or end of a clip. It aims to enhance synchronization with audio or emphasize certain moments in the video.
Image Credits: Adobe
To address concerns around deepfakes and authenticity, Adobe plans to introduce Content Credentials into Premiere. These metadata tags will confirm if media was AI-generated and specify the AI model used for its generation.
Regarding data training for the model, Adobe did not disclose details on the sources of training data, including images or videos used. There was no mention of compensation arrangements for contributors either.
Recently, reports indicated that Adobe is paying photographers and artists on Adobe Stock to provide video clips for training its video generation model, offering varying compensation rates based on the quality and content submitted.
Unlike some competitors using publicly available data for model training, Adobe seems focused on ensuring proper licensing and ownership of training data to avoid legal issues. It aims to position itself as a reliable option with its IP indemnity policy for enterprise customers.
Details on pricing for the new video generation features in Premiere are not yet disclosed. Adobe plans to continue its generative credits system, where subscribers receive monthly credits based on their plan to use the generative tools.
While these advancements are promising, questions remain about the effectiveness and value of Adobe’s AI-powered video capabilities. The company’s previous image generation models faced criticism, and the lack of a clear release date for the video model raises uncertainties about its success.
Adobe’s collaboration with third-party vendors like OpenAI signals potential future integrations to enhance Premiere’s video generation capabilities. These partnerships are currently in the early stages and not a finalized product offering.
Adobe acknowledges the need to stay competitive in the generative AI landscape and is exploring avenues to advance its technologies further. However, with evolving competition and consumer expectations, the company will need to deliver compelling results to stand out in this rapidly developing field.
