Adobe Answers the Sora Question & New SD3 Features!
TLDRAt the NAB convention, Adobe unveiled new AI video generation features for Premiere, including object removal and addition, as well as generative audio extensions. Microsoft introduced Vasa, an AI avatar with lifelike, audio-driven talking faces. The first AI art esport competition showcased the potential of community events centered around generative AI technology.
Takeaways
- 🌟 Stable Diffusion 3 has been released, offering improved prompt understanding and text generation within images.
- 🔍 The new version includes features like a creative upscaler up to 4K, in-painting, out-painting, search and replace, and background removal.
- 📈 Stable Diffusion 3's image-to-image with a mask allows for modifications using an image prompt, enhancing creative control.
- 🖼️ A comparison between Stable Diffusion 3 and XL shows SD3's superior clarity and detail, especially in complex patterns and textures.
- 🤖 Microsoft has entered the AI avatar market with Vasa 1, which generates lifelike, audio-driven talking faces in real time.
- 🎥 Adobe is integrating generative AI video features into Adobe Premiere, focusing on object removal, addition, and generative extend for video editing.
- 📢 Adobe's Firefly video model aims to be commercially safe and will provide content credentials for generated media, ensuring transparency.
- 📚 Adobe is working on an ideation environment on adobe.com for users to experiment with video models and provide feedback.
- 🎨 Adobe Premiere will receive smart masking and improved manual masking tools, leveraging existing technology for better object selection and rotoscoping.
- 📈 The first AI art esports competition showcased the potential of community events centered around AI-generated art and real-time drawing tools.
- 📝 Adobe's commitment to an open ecosystem suggests that third-party plugins and models will continue to play a significant role in their software offerings.
Q & A
What is the main topic discussed in the video?
-The main topic discussed in the video is the release of Stable Diffusion 3, its features, and a discussion with Adobe about new AI video generation features coming to Premiere Pro.
What are some of the new features of Stable Diffusion 3?
-Some of the new features of Stable Diffusion 3 include prompt understanding, text generation within images, a creative upscaler up to 4K, in-painting, out-painting, search and replace, background removal, and image-to-video conversion.
How does Adobe's new AI video generation feature in Premiere Pro work?
-Adobe's new AI video generation feature in Premiere Pro allows for object removal, object addition, generative extend to match the beat of music, and generative audio along with clip extensions.
What is the significance of the AI art esports competition mentioned in the video?
-The AI art esports competition signifies the growing potential for community events based around AI-generated art. It involves contestants using real-time drawing tools to create images based on prompts, showcasing the interactive and competitive aspects of AI art.
What is the difference between Stable Diffusion 3 and Stable Diffusion XL?
-Stable Diffusion 3 is reported to be better at prompt understanding and generating more detailed images, such as more celestial patterns on robes and clearer staff details, compared to Stable Diffusion XL.
What is the role of Adobe's Firefly image model in the development of their video model?
-Adobe's Firefly image model is a collection of AI models that the company is using as a foundation for developing their video model, which will be announced later in the year.
How does Microsoft's Vasa AI avatar differ from other AI avatar models?
-Microsoft's Vasa AI avatar is audio-driven, meaning it generates realistic lip sync and facial expressions based on actual recorded audio, as opposed to text-based models.
What is the purpose of the content credentials attached to media generated through Adobe tools?
-Content credentials serve as a 'nutrition label' for AI-generated media, indicating whether the media was fully AI-generated or just modified, and which model was used in its generation.
What is the significance of the 'Sizzle Reel' mentioned in the discussion with Kyle from Adobe?
-The 'Sizzle Reel' showcases the capabilities of Adobe's current models, demonstrating how they can generate high-quality content, with the promise of continued improvement and increased control over time.
What is the Stable Assistant Beta platform mentioned by the speaker?
-Stable Assistant Beta is a new platform announced by Stability AI, which is described as a friendly chatbot that allows paying subscribers to access the latest models, generate images, write content, and match photos to text through conversation.
How does the AI art esports competition work?
-The AI art esports competition involves contestants using Leonardo's real-time drawing tool to generate an image based on a randomly selected prompt within one minute, making it a fast-paced and exciting event.
Outlines
🚀 Stable Diffusion 3 Release and Features
The video discusses the recent release of Stable Diffusion 3, an AI model that excels at prompt understanding and generating text within images. It highlights new features such as a creative upscaler for images up to 4K, inpainting and outpainting without a mask, search and replace functionality, background removal, and image-to-video conversion. The video also compares Stable Diffusion 3 with Stable Diffusion XL through various examples, showcasing the improved crispness and detail in the newer version. Additionally, Stability.a has announced a new platform called Stable Assistant Beta, a chatbot providing access to the latest models for a subscription fee.
📽️ Adobe's Generative AI Video Features in Premiere
The script covers a conversation with Kyle from Adobe about the upcoming generative AI video features in Adobe Premiere. These features include object removal, object addition, generative extend, and matching room tone. Adobe is working on a video model that will be announced later in the year and is exploring collaborations with major video AI model providers. The video also discusses the potential for a standalone platform for video within the Adobe ecosystem and the importance of content credentials for AI-generated media. The conversation ends with a mention of generative audio and the possibility of improved manual and smart masking tools in Premiere.
🎭 Microsoft's Vasa 1: Real-Time AI Avatars
The video introduces Vasa 1, Microsoft's entry into the AI avatar space, which is audio-driven and capable of generating highly realistic and expressive talking faces. The technology is showcased through examples where characters display nuanced facial expressions and head movements that contribute to a sense of authenticity. The video also notes the ability to control camera angles and text prompts within Vasa 1, suggesting potential applications in virtual meetings and presentations.
🎮 First AI Art Esports Competition
The video concludes with a discussion about the first AI art esports competition, in which the speaker served as a judge. Organized by Creative Refuge, the event involved contestants using Leonardo's real-time drawing tool to create images based on prompts given by the audience. The competition was described as exciting and demonstrated the potential for community events centered around AI technology. The speaker encourages viewers to check out the full rundown of the event on Creative Refuge's channel.
Mindmap
Keywords
💡NAB convention
💡AI video generation features
💡Stable Diffusion 3
💡Microsoft AI avatar
💡Content credentials
💡Firefly image model
💡Stable Assistant Beta
💡AI art esport competition
💡Smart masking
💡Adobe Dynamic Link
💡Scenario platform
Highlights
Stable Diffusion 3 has been released with improved prompt understanding and text-to-image generation capabilities.
Stable Diffusion 3 offers a creative upscaler that can upscale images up to 4K resolution.
The new feature 'search and replace' allows users to replace objects in images using simple language prompts, without needing a mask.
Stable Diffusion 3 includes a built-in background removal feature for convenience.
Image-to-video feature connects directly to Stable Diffusion Video, enhancing creative possibilities.
Stable Diffusion 3 introduces image-to-image editing with a mask, allowing for text or image prompts to modify specific parts of an image.
Comparison of Stable Diffusion 3 and Stable Diffusion XL shows SD3's improved clarity and detail in image generation.
Adobe discusses the integration of generative AI video features into Adobe Premiere, including object removal and addition.
Adobe's Firefly video model aims to provide commercial safety and will attach content credentials to generated media.
Microsoft enters the AI avatar game with VASA 1, an audio-driven, lifelike talking face generation technology.
VASA 1's lip sync and facial nuances contribute to the authenticity of AI-generated characters.
The first AI art esports competition was held, showcasing the potential for community events around AI technology.
Adobe is working on bringing more control and granularity to their AI models, enhancing user experience.
Adobe Premiere will integrate smart masking and improved manual masking tools for better object selection and rotoscoping.
Stable Assistant Beta, a friendly chatbot, provides subscribers access to the latest models for image generation, content writing, and photo-text matching.
Adobe's open ecosystem approach allows third-party plugins to serve specific niches within AI, offering customers choice.
The AI art esports competition demonstrated the excitement and potential of real-time AI drawing tools for community engagement.
Adobe's exploration with video AI model providers like Open AI, Runway ML, and Pika aims to enhance Premiere Pro's capabilities.