And there we go. We got a giant
cat. Yes. Okay. I didn't know what I expected. Hello and welcome to Stable
Diffusion Experimental. Today, we're going to be looking at how to use Stable Diffusion 3
via API key in CompUI. Now, a quick disclaimer, Stable Diffusion 3 has been released to use
with API keys, but it hasn't been released as a checkpoint. What does this mean to you? It
means basically that SD3 right now can only be used with API calls and as such, it's not free
to use. In order to use it, you need to buy some credits. The credits are going to cost you around
6 cents of US dollar for image generation. And if you want to wait for Stability AI to release the
free to use checkpoint in the future, then you can absolutely do so. But in the meantime, if you
want to use it with comfyUI there's a way to do that. And I'm going to guide you through the whole
process of setting up comfy workflow and setting up your API keys and then generate the images. So
there's a few different ways to set up a workflow in ComfyUI that can use SD3 right now. But since
Stable Diffusion is released by Stability AI and Stability AI has released a workflow and nodes
on their own GitHub, I'd rather use those. So we're gonna head over to Stability AI GitHub page
for ComfyUI SAI API. Here we can see that GitHub has been updated yesterday and we can see an
image of a cat wearing a hat with API on top and Stability on its shirt. This image contains
the workflow so in a second we're going to drag and drop this image into a ComfyUI workspace.
First, let's check the requirements. We can see under usage that we need to add an API key.
So we're going to need that as well. But first, let's save this image or drag it and drop it into
a comfyUI workspace. So we drag and drop it into our comfyUI workspace and we can see that we have
a lot of missing nodes. So the first thing we're going to do is go over to manager and then we're
going to install missing custom nodes. We're going to click on it and we're going to see Stability
API nodes for comfyUI that's missing. We're going to click on install and after it's done installing
it's going to ask us to restart comfyUI. We're going to click on restart button and comfyUI is
going to tell us that it's reconnecting to the instance. Once comfyUI has finished restarting, we
can refresh the page and we're going to be greeted with the new nodes we have now. So now we're going
to inspect them one by one, starting from the left, we have stability image core, Then we have
stability SD3. This node connects to various other nodes. We have a preview image node for an image
generator with stability SD3. We have stability remove background, which previews an image with
the removed background. We have stability creative upscale and a preview image node for that as well.
Then we have stability outpainting and the preview image node as well. Then on the bottom we have
stability inpainting which feeds from a load image node and outputs a preview image node. And
the last node we have is a stability research and replace node that outputs a preview image. Now
before getting into what these nodes actually do we have a single thing that appears in every
single node. And that is API key override. In this field, we need to input our own Stability AI API
key. So now I'm going to show you how to get that and get some credits in order to generate images
with stability AI SD3. So right here on this webpage, you will find the link in the description
below. We have all the documentation for stability AI newest SD3. If we click on pricing here, we
can see all the different pricing for all the different models they have available to use via
API key. We can see here for SD3 it's going to take 6.5 credits per API call while SD3 turbo is
going to take 4 credits, while Core is going to take 3 credits. When you compare it with SDXL 1.0
or SD 1.6, it's going to take a lot more credits than those, as the XL costs between 0.2 and 0.6
credits for example. So how do we get credits? Well, if we look over here at buying credits, we
can see that we have an initial 25 free credits but additional credits can be purchased. And the
credits are priced at $10 for a thousand credits. In order to purchase them, we need to go over to
our account page. If we click on the link, we are greeted with our welcome screen, and if you don't
have an account, you can sign up otherwise you can just log in. So right now in my account page, I
have 25 credits and I can buy some more for $10. Please mind that $10 is the minimum amount to buy.
If we put in like $5 and then click, we're going to see that the amount reverse to $10. On the
other hand, we can buy more. So if we input $20, for example, we're going to buy 2,000 credits. So
if we click on the buy button, we are going to be greeted with a pay with card page. Now, of course,
I'm not going to show you my personal information. I'm not going to fall for that. But if you
want to buy some more credits, instead of the 25 you have available at first, you can do that
over here. Now going back to our account page, we can see that this API keys here on the left. If
we click on that, we can see that we have API keys over here. You can click on the icon to reveal
the API keys. Again, I'm not going to fall for that. And we can copy them to the clipboard.
So we're going to copy those to the clipboard and we're going to back to our ComfyUI window. Now
back to the ComfyUI window, we want to input that key into our API key override field. So click on
that, control, read a key, and click OK. Please note that you will have to do that for each single
node that you want to use. But if you don't care for some nodes like e stability search and replace
node, you can just either delete it or just bypass it. So let's start with core and SD3. Core and SD3
refer to the two models that were available for pricing at the web page that I showed you just a
second ago. Core appears to be a more basic model, while SD3 appears to be a more refined model.
Inside the core node, we can find a positive prompt field, a negative prompt field, a seed, a
control after generate field, which is a random fixed incremental, etc. An output format,
which right now is PNG and an aspect ratio. In the SD3 node, instead we can select a model
and we can select a model between SD3 and SD3 turbo. If you're worried about credits, SD3
turbo is slightly less costly than SD3. And over here as well, we can see a positive prompt
field, a negative prompt field, a seed, a control after-generate field, strength, aspect ratio, and
output format. Now the weird thing is that here, strength, aspect ratio, and output format are kind
of scuffed. I haven't changed anything since I installed it via the ComfyUI Manager, so I'm going
to try and make sense of it after running the first generation. You can see here that the output
format should be PNG, like in the core node, but instead it's 16:9, which should be in the aspect
ratio field. So my guess is that the input is one field lower than it should be. It makes sense
for 16:9 to be under aspect ratio and it makes sense for 0.8 to be under strength. So let's try
and fix that. Let's put 0.8 here in strength and 16:9 here in aspect ratio. Let's see under output
format and there it is we have PNG or Jpeg. So yeah if you find this issue yourself you can just
fix it like that. So for first test let's disable everything that is on core or SD3 by selecting
bypass on every single node that is in core or SD3 and there we go. Right now we have only core and
SD3 active. We are gonna input a positive prompt let's say we want a passion photography of a young
woman wearing Miu Miu haute couture in a baroque room filled with sunlight coming from a skylight.
We're going to leave the negative prompt field empty because I want to see how it works with
positive prompt field only and we're going to copy the positive prompt field over to our SD3 node as
well. I'm going to put both previews side-by-side so that I can compare them and I'm going to
hit Queue prompt. And there we go. We got our images free of charge because we had 25 credits
available. Before delving into these images, let's go back to our stability AI account page and
check the status of our credits. And there we go, if we go into the billing page, we can see that we
now have 15.5 credits remaining. So for these two generations, we used 9.5 credits. Now let's look
at these images. Now from a fashion photographer point of view, the dress on the left is not really
Miu Miu. There are some things in the prompt that influenced it like Baroque. It seems like the core
model kind of translated the environment settings into the clothing and general mood as well. But if
we look at the image on the right hand side which is made with SD3, we can see that the clothing
is much more similar to Miu Miu look and feel. Also prompt the earrings is much stronger. We
actually have a skylight in the right picture, whereas there is no skylight to be seen in the
left picture. Both images honestly are really good, especially when you realize that this is a
base model. We don't have fine tune models made by the community yet. We don't have no juggernaut,
we don't have any RealVIS. We don't have anything else, but a model that can be called via API. So
honestly, as a literal first impression for me, because this is the two first images
that I generated using these models, big thumbs up. Now I don't want to be the one who
says back in my day, but back in my day as the XL and as the 1.5 as base models were much worse
than this, at least from what I remember. So now let's go inspect all the other notes that we left
out right now. We have this stability creative upscale. We're gonna click on bypass to activate
it since it was first bypassed. We're gonna copy the prompt from the previews node and we're gonna
input right here. I'm gonna leave it just like that. I'm gonna input my key over here and I'm
going to leave it just like that. I'm going to input my key over here and I'm going to hit Queue
prompt. And there we go. Honestly, pretty amazing result right off the bat. Yes, the dress is being
influenced a lot by the Baroque setting and it's not the same dress as before. We can fix that.
I guess, by lowering the creativity setting, but the detail in there. My god, for a base model this
is fantastic. It's corrected a lot of the anatomy. The lighting is amazing and the intricate details
of the dress and of the environment are absolutely stunning. Even the background and the city outside
the window. It's absolutely stunning. The shadow here is wrong, but apart from that I can see
almost no issue. So now let's try out painting. I am going to disable the upscale node just
in case. It shouldn't recalculate anything since the seed is fixed, but I don't want to risk
spending any credits or at least any more credits that I need to. So I'm going to right click and
click on bypass. Now I'm going to copy again the prompt from the previous node and paste it over
here and I'm gonna copy the API key and paste it in the API key override field. Let's hit Queue
prompt and see what happens. And there we go. It's a bit scuffed. I'm not gonna lie. It changed
the perspective a lot and maybe it was too much of an outpainting. 512 in all directions, it's maybe
a bit too much. So let's do another experiment. I am gonna re-enable the upscaler node and I'm
gonna feed it into the outpainting node. And next I am gonna do 256 on the left and 256
on the right and I'm gonna zero down on up and down. So it should just expand the image to
the sides. That's it, Queue prompt again and see what happens. And unfortunately we have countered
an error. The payload cannot be larger than 10 MB in size. It appears that our upscale image is too
large to be handled by the outpainting node. Not a big deal, we can just relink the SD3 output to the
outpainting node and rerun the prompt. And there we go, outstanding result. The perspective is much
the same, the ambience is the same, we got some nice light coming from the windows, we got some
nice glare effect in the lens, this is amazing. So since before the payload was too large coming
from the upscale image. Let's upscale this one instead. So I am going to link the output from the
outpainting node into the upscaling input. And I'm going to hit cube prompt again. And there we go.
We got much more detail in the outpainted areas right now. And honestly, amazing. I'm really happy
with this kind of results again mostly because this is a base model. Alright we tried
the core model we tried the SD3 model we tried upscaling we tried out painting. Now
let's try inpainting and then search and replace. So I'm gonna right click and click on
bypass to enable stability in painting mode So I'm going to right click and click on bypass to
enable stability in painting mode and I'm going to copy the API key inside of this field. Then since
it's asking for both an image and a mask, then since it's asking for both an image and a mask, I
am going to input the image that we just created and I am going to open it into our mask editor. So
I'm going to input the image that we just created and I am going to open it into our mask editor.
So I'm going to right click and select open in mask editor. Here I am going to select the face
and roughly the whole body. And I'm going to ask for a quick change of clothes and for a different
model. So I'm going to ask for a young Asian woman and side note I would really like if the default
for young woman will be white basically all the time and I'm gonna add wearing Balenciaga
ready to wear couture. I'm gonna hit Queue prompt and we're gonna see how that works. And
there we go. We got a nice change of clothes, a nice change of models. We got some remnants
from the previous picture. For example, it did not care for the hair. It transformed
it into a sort of crown and we got some blonde hair in the background as well. So some
issues there, but overall, I didn't want to select the hair strands on top of a head to
see what it could do with those. And overall, yeah, it might be a masking issue, it could be
a user error, it could be that the in painting model is kind of struggling with those things, but
everything else is really good. So now let's try the last note. We're gonna enable the search and
replace node by clicking on bypass, and gonna use this ability SD3 image as our input. And honestly,
I'm gonna leave it just like the default. I really want to see how it can substitute a person with
the cats in this kind of situation. So I'm going to enable the API keys and hit Queue prompt. And
we got an error. It's expecting JPEG PNG or WebP, but it received nothing. So why is that? Well, PNG
should have been in the output format, I guess. So I am going to select it over here. And I am going
to delete this PNG and see what happens over here. Oh yeah, that's an active prompt field. It appears
that the PNG format has been pushed into an active prompt field and that was the error. So if you're
trying that, remember to delete PNG from that field and enable it in the output format field.
So last hit queue prompt again and there we go. We got a giant cat. That's okay. I didn't know what
I expected. That wasn't it. I guess the person was replaced and the perspective is not quite the
same or maybe it's a very tiny house who knows but the cat is very good. And also the lightning is
kind of consistent. So could it be better? Yes. Is it funny? Yes. So that was basically it.
These are the nodes that we have available from the official GitHub page from stability AI.
The only note that I couldn't get working is the remove background node since it kept asking me
for an API key, but there was no field to input it. There might be a workaround by creating a text
file inside the folder that holds your API keys, but I'm not going to try that because I don't
want to break anything. I'm sorry if I'm still kind of laughing, but the cats staring at me in
from the right corner and Look at that cat it's humongous. But yeah, if you want to try SD3 you
know now how to do it inside of comfyUI you don't have to rely on any external third party web
interfaces, you can just run it on your ComfyUI workspace as you're used to and even if you don't
want to pay anything, you still got 25 credits to use. So those credits should run you at least a
couple of generations, and you can test it out. I still have some credits available, so if you
want to leave some comments with some prompts, I can try and test those out for you and I will
leave links to let's say imgur where you can see the resulting images and we can all test it out
together. That's gonna be all for today my name is Andrea Baioni you can find me on Instagram
@risunobushi or on the web at andreabaioni.com. And I'm gonna leave you with a slideshow of SD3
images that I just generated. See you next time.
