Stable Diffusion 3 via API in comfyUI with Stability AI official nodes - SD Experimental
TLDRIn this video, Andrea Baioni guides viewers through the process of using Stable Diffusion 3 (SD3) via API key in ComfyUI, a platform for generating images using AI. He explains that while SD3 is not free and requires the purchase of credits, Stability AI provides an initial 25 credits for users to test the service. The video demonstrates how to set up the workflow in ComfyUI, install necessary nodes from Stability AI's GitHub, and input the API key for each node. Baioni also provides a step-by-step tutorial on generating images using SD3, including core and SD3 models, and explores additional features like upscaling, outpainting, and inpainting. He shares his first impressions of the models, noting the quality and detail of the generated images, and invites viewers to leave prompts for him to test. The video concludes with a slideshow of SD3 images generated during the tutorial.
Takeaways
- 📈 Stable Diffusion 3 (SD3) is available for use via API key but not as a free checkpoint, requiring users to purchase credits for image generation.
- 💵 The cost for image generation with SD3 is approximately 6 US cents per image, with a choice between SD3 and SD3 Turbo models.
- 🚀 Users can utilize SD3 with ComfyUI by installing the required Stability AI nodes from the official GitHub repository.
- 🔑 An API key is necessary for using SD3, which can be obtained and managed through the Stability AI account page.
- 🎨 The script demonstrates various nodes such as image core, SD3, remove background, creative upscale, outpainting, inpainting, and research and replace, each serving different image manipulation functions.
- 🌐 The API key must be entered for each node individually in the ComfyUI workflow.
- 👗 A positive prompt example given was 'a passionate photography of a young woman wearing Miu Miu haute couture in a baroque room filled with sunlight.'
- 📸 The output images from the core and SD3 models were compared, with SD3 producing more accurate and detailed results.
- 🖼️ Additional features like creative upscale and outpainting were tested, showing impressive results, albeit with some adjustments needed for optimal use.
- 🧩 The inpainting node was used to change elements within an image, such as altering clothing and introducing new subjects.
- 🔍 The search and replace node was tested, resulting in a humorous outcome where a person was replaced with a giant cat in the image.
- 🛠️ The remove background node was not functional in the demonstration, possibly due to a missing field for the API key input.
Q & A
What is the current status of Stable Diffusion 3 (SD3)?
-Stable Diffusion 3 has been released for use with API keys, but it is not available as a free checkpoint. It can only be used via API calls and is not free to use.
How much does it cost to generate an image with SD3?
-It costs around 6 cents of US dollar for image generation using SD3.
What is the process to use SD3 with ComfyUI?
-To use SD3 with ComfyUI, you need to install the missing custom nodes from the Stability AI GitHub page, add an API key, and then generate images using the nodes provided.
What are the different nodes available for SD3 in ComfyUI?
-The nodes available for SD3 in ComfyUI include stability image core, stability SD3, stability remove background, stability creative upscale, stability outpainting, and stability inpainting.
How can one obtain Stability AI API keys?
-To obtain Stability AI API keys, one needs to sign up or log in to their account on the Stability AI website, navigate to the account page, and then click on the API keys section to reveal and copy the keys.
What is the pricing for using different models via Stability AI's API?
-The pricing varies by model: SD3 costs 6.5 credits per API call, SD3 turbo costs 4 credits, and Core costs 3 credits. SDXL 1.0 or SD 1.6 cost less, ranging between 0.2 and 0.6 credits.
How can one purchase additional credits for Stability AI?
-Additional credits can be purchased by going to the account page, selecting the desired amount (with a minimum purchase of $10 for a thousand credits), and then proceeding to pay with a card.
What is the output format for the images generated by the SD3 model in ComfyUI?
-The output format for images generated by the SD3 model in ComfyUI is initially set to PNG, but it can be changed to JPEG if desired.
What is the purpose of the 'positive prompt' and 'negative prompt' fields in the SD3 node?
-The 'positive prompt' field is used to guide the image generation process by providing specific details or themes desired in the image. The 'negative prompt' field allows users to specify elements that should be avoided or not included in the generated image.
How did the speaker evaluate the initial images generated by the SD3 model?
-The speaker was impressed with the initial images generated by the SD3 model, noting that the clothing and environment were well-translated from the prompt, and the results were good considering it was a base model without community fine-tuning.
What issue did the speaker encounter when trying to use the 'stability search and replace' node?
-The speaker encountered an error when using the 'stability search and replace' node because the output format was mistakenly pushed into an active prompt field instead of being set in the output format field, causing the node to expect a JPEG, PNG, or WebP format but receive nothing.
What was the speaker's final verdict on using SD3 within ComfyUI?
-The speaker concluded that using SD3 within ComfyUI is a viable option and does not require reliance on external third-party web interfaces. They also mentioned that the 25 free credits provided are sufficient for a few generations of image generation to test out the model.
Outlines
📝 Introduction to Stable Diffusion 3 and CompUI API Key Setup
The video begins with an introduction to Stable Diffusion 3 (SD3) and its use via API key in CompUI. It is clarified that while SD3 is available for use with API calls, it is not yet released as a free checkpoint, meaning it requires the purchase of credits for image generation, costing approximately 6 cents per image. The video promises a walkthrough on setting up the workflow in CompUI, including API key configuration and image generation. The process involves using a workflow and nodes provided by Stability AI on their GitHub page, addressing the installation of missing custom nodes, and the need for an API key for usage.
💳 Purchasing Credits and Setting Up API Key in CompUI
The speaker guides viewers on how to acquire Stability AI credits and set up the API key for SD3 in CompUI. After purchasing credits, which start with an initial 25 free, the user is shown how to navigate to the account page to reveal and copy the API key. Each node in the CompUI workflow requires the API key to be manually entered. The video also covers the selection of models within the SD3 node, including the cost-efficient SD3 turbo option, and the process of inputting prompts for image generation. A test run is demonstrated, generating images using the core and SD3 models, with a comparison of the results and a check on the remaining credits after generation.
🖼️ Exploring Additional SD3 Nodes for Image Manipulation
The video continues with an exploration of other Stability AI nodes available for image manipulation within CompUI. These include the stability creative upscale, outpainting, inpainting, and search and replace features. Each node is tested with a specific prompt, and the results are discussed. The speaker notes some issues with the outpainting node due to payload size limitations and provides a workaround. The inpainting node is used to change the subject of an image, and the search and replace node is tested for replacing a person with a cat in an image. The results of these experiments are shown, and the video concludes with a positive outlook on the capabilities of the base SD3 model.
🚀 Conclusion and Future Testing with SD3 in CompUI
The video concludes with a summary of the SD3 nodes tested and an invitation for viewers to suggest prompts for future testing. The speaker mentions the inability to get the remove background node working due to a missing API key input field but does not attempt a workaround. The video ends with an offer to test prompts provided by viewers and provides links to view the generated images. The speaker introduces themselves, provides social media handles, and concludes with a slideshow of SD3 images generated during the video.
Mindmap
Keywords
💡Stable Diffusion 3
💡API key
💡ComfyUI
💡Image generation
💡Credits
💡Positive prompt
💡Negative prompt
💡Upscaling
💡Outpainting
💡Inpainting
💡Search and replace
Highlights
Stable Diffusion 3 (SD3) is now available to use via API keys, but not as a free checkpoint.
Using SD3 requires purchasing credits, costing around 6 cents USD per image generation.
Stability AI's official nodes and workflow for ComfyUI are available on their GitHub page.
Missing nodes in ComfyUI can be installed via the manager to integrate SD3.
Each Stability API node in ComfyUI requires an API key override for authentication.
Stability AI offers different pricing for various models available via API, with SD3 costing 6.5 credits per call.
Users start with 25 free credits and can purchase additional credits in increments of $10 for 1000 credits.
The API key must be manually entered into each node in ComfyUI for it to be functional.
Core and SD3 models differ in complexity, with Core being more basic and SD3 more refined.
Positive and negative prompts can be used to guide the image generation process in SD3.
The output format and aspect ratio fields may require manual adjustment in the SD3 node.
The generated images from the Core and SD3 models showcase the models' ability to interpret prompts and create detailed scenes.
Stability Creative Upscale node enhances image details and corrects anatomy with impressive results.
Outpainting node expands images while maintaining perspective and ambience, though large expansions may result in errors.
Inpainting node allows for changes in the image, such as altering clothing or models, with some limitations.
Search and Replace node can substitute elements within an image, as demonstrated by replacing a person with a cat.
Remove Background node could not be tested due to a lack of API key input field.
ComfyUI users can utilize SD3 without relying on external interfaces, using the provided 25 free credits for testing.
Andrea Baioni, the presenter, invites viewers to share prompts for further testing and exploration.