Beginner's Guide to Stable Diffusion and SDXL with COMFYUI

hey guys this is Kevin for Pixel foot and in this video we're going to be taking a look at stable diffusion sdxl stable diffusion extra large and also comfy UI and the best way of getting started is to really show you some of the images that I created with the software these were created in sdxl stable diffusion XL using comfy UI and with sdxl it's capable of producing images like this as soon as you install it you don't need to get third party this third party that so these were produced with the standard model that comes from stability Ai and the 1024 by 1024 in most cases you can produce larger than that and you can see there's a very wide variety of different types of images now the images come from prompting and you can see some somewhat photorealistic images and some images that are complete fantasy this one is one of the complete fantasy ones and I could never have designed anything and drawn anything this spectacular myself but with the software you can do that it's really quite amazing all it takes really is just some text prompts to get you going this one is completely different still fairly surrealistic but very minimalistic and um in some ways it's kind of minimalistic ones are a little bit more tough uh to do sometimes this is one which is fantastic in that it's somewhat photorealistic but it's also completely completely invented this one is kind of like a statue it looks almost like a photograph so these aliens were prompted with the same prompt you can get this despite using the same prompt this one is a very different one which looks fairly photographic it looks maybe like the bronze statue and I think it's almost instantly recognizable another one that I really like this one wasn't what I was expecting but I thought it was pretty good and very difficult to do is to get a good hand inside of the software now with this one I was trying to tell the software it looked produce an image with an alien standing on top of a hill I want to see a city city-sized spaceship in the distance and it got it it really did get it and some very difficult things to produce are realistic faces models uh this one's a statue but I managed to do it and it's uh with a little bit of a sense of achievement that we managed to get something like this and uh I think this is actually it looks fairly detailed and it looks almost like uh almost like a photograph and this final image here is from stability AI themselves these are some of the pictures that they chose I really like this one it looks almost like a photograph taken by a professional um now that we've seen the end of all of this let's go back to the beginning good place to start is the stability AI account on hugging face now it's a good idea probably to have an account here you can create an account for free and you can sort of favorite pages and you can favorite files we're going to be downloading some files from the from stability AI who are really the guys who started all of this for sdxl for stable diffusion XL you need three files we'll take a look at those in detail but there are also files here for 1.5 stable diffusion 1.5 and also for 2.1 so if you come here it's going to open up that in the background that will give you stable diffusion 2.1 that is one which is not the most popular one I have to say the older versions are available here on this website but you'll find those with a different company so if we say stable diffusion a lot of people prefer stable diffusion 1.4 let's try that again [Music] what you'll find is that it's being offered by a company called Compass now this is the guys who really were the beginning of this it was even before uh stability AI this is the original and some people prefer still to work with this I I think it's a good idea to download this I don't think the models they provide necessarily are the best modules in the world but some people will use these ones and um version 1.4 that's the checkpoint file you could also try a different one as well there's a different version I think I use the the standard one there but my preferred one is actually 1.5 stable diffusion 1.5 so stable diffusion is actually open source and that means the project has been taken by different people and used in different ways comfists were probably the very first people to start Distributing this but there's another company there's another organization called Runway ml stable diffusion 1.5 there we are just type in Runway ML and you can come to this download page you can see I think the downloads have sort of gone off a cliff for this particular file probably because it's the most popular version of stable diffusion and sdxl the new version has come out I think that would explain it that sudden drop in popularity but this one coming from Runway a very good very very good version and there are two versions here one is the pruned version and another one is the unpruned version which is apparently requires more vram I've never used it but it's suitable for fine tuning which basically means you can use this one for training a lot of people want to train the software they want to train the stable diffusion to do specific things that the base models don't do and yeah so this one is the better better one for that I would definitely recommend this one because um you might want to compare with sdxl when you start using it sdxl but as you Excel itself well before stxl came stable to Fusion 2.1 this is the one which is not really that popular but you can download it here they've got the licenses the old limitations and biases and this one was not very popular so if you go to the files and and versions you will find different versions and you can decide for yourself which one you should go for probably when you have an option the safe tensor one is the one that you should go for it will work with comfy UI it will also work with other popular versions of stable diffusion [Music] I think automatic 11 11 the ckpt ones are good files if they're coming from a reputable Source like stability AI but they can contain code that you don't really want running on your computer so you really wouldn't be downloading this from from you know some random dodgy website which you know nothing about you probably want to go with the safe tensors which are not gonna randomly execute code on your account and you suddenly find your bank account emptied by some gentleman in Moldova so the ckpt ones from companies that you absolutely trust safe tensors from any place where you're not entirely sure about what's happening but this one 2.1 honestly I've never used it and you might give this one a miss we will take a look at sdxl sdxl is a little bit more complicated it's a little bit more complicated we'll take a look at that separately what I want to do is to take a look at a couple of files that you're almost certainly going to need to install on your system now I know a lot of people probably are already using SD um 1.5 but if you're not you almost only need to install the software python 3.10 this is going to be useful pretty much all the time when you're running AI related software the one for me would be the windows installer one that's the recommended one you probably want to install this guy here there's nothing yeah install this guy and get for Windows download this one it will take you to the latest version and for for those people who haven't subscribed to one of my courses uh over at udemy there are some courses that you can subscribe for um there's one for comfy UI which we're going to be using so over three hours of material there so it's very in-depth then goes into a lot of very advanced stuff uh it is cover stable diffusion comfy UI sdxl the new version of stable diffusion and because config UI is the version of stable effusion that is the most suitable for sdxl this one I've been updating quite regularly I also have a version for automatic 11 11. this is an introductory version and this is probably the most popular version of stable diffusion automatic 11 11. now automatic 1111 doesn't use the Ensemble of experts method which was the advised by Nvidia and that method is the basis for sdxl so they have been updating it but it hasn't yet implemented these Ensemble of experts method so I'm probably not going to be updating this course till you know they've implemented The Ensemble of experts method what I'll do for all of these courses I'll give anyone here who's watching give you guys a discount if you want to go over and uh register for those so do that as soon as you can because I don't think the discounts will necessarily last forever but the one I would definitely recommend is the coffee UI one if you want to learn about sdxl now sdxl came out just a couple of weeks ago you might need you'll need three files minimum you might have already downloaded the sdxl vae safe pencil you're going to need that one if you haven't downloaded it if you have downloaded it download it again because it's been recently updated and what you'll need to do just you click on this button you'll need to download it again because there was a problem with the original one so it's important that you get updated that is probably a top priority you'll need this file as well stable diffusion Excel refiner so there is what they call The Ensemble of experts method it requires you to use a couple of models of stable diffusion and this is the second model that is used in the sequence uh it's the one where we looked at the pictures that he produced the method is described here and The Ensemble of experts pipeline is described there but what I really want to look at is the evaluation this was done by stability AI themselves and in their evaluation they found stability diffusion 2.1 to have the lowest score 1.5 to have uh a very low score score the beta version which came out fairly recently actually much higher but the favorite versions were the new versions that's the sdxl 1.0 that's the base and refiner that's using the Ensemble of experts method and sdxl 1.0 just the base one produces almost as good a result as the beta base and refiner so you could use sdxl the base one just on its own or you could combine it with the refiner to get even better results now um what they say here is important to understand I'm just going to bore you a little bit with some of the details they give here now but this stuff is important so let's take a look there is a intended use section which you probably want to read out of scope so they say you cannot use it it was not trained to be factual or to or to produce true representation of people or events important to read the limitations the model does not achieve perfect photorealism it cannot render legible text you can save yourself a lot of time by not trying to render legible text inside of this model the model struggles with more difficult tasks which involve compositionality such as rendering an image corresponding to a red cup or a Red Cube on top of a blue sphere you'll get blue everywhere or red everywhere no some of the images that we saw around at the beginning I didn't choose those randomly you could see some of the images there was a lot of red everywhere and they look Splendid but it does struggle to contain color to exactly where you want that color to be faces and people in general may not be properly generated so this is something I'm working on on the course that I describe the coffee UI course where we're trying to get better quality people better quality results the auto encoding part of the model is lossy and there are obviously some biases I'm not sure what that means they're building encoding part of the model is lossy you want to download SD stable diffusion Excel base 1.0 so they do call it sdxl and stable diffusion XL the base one is probably the more important one and that's the one you're going to be using Pro probably some people might actually use just this one and again we go to the files okay there are more files which have been added I think you would download this guy here the one I'm using is this guy here and I think they've updated it fairly recently as well another website where you almost only want to go to especially if you're using SD 1.5 is a Civic AI or civitay this website has a couple of models which you could use instead of say the 1.5 model from stability AI so with this one it's base model SD 1.5 so it's only a couple of gigabytes it's a little bit smaller than the one from hugging face again you could sign in for this one you could create an account and that will allow you access to more stuff including some of the more not safer work models that they have here so with this one I would definitely download this one it was updated to number seven quite recently it's been updated to number eight so I'm gonna be downloading this one we're gonna be using number seven later on there's also another one that you should definitely consider which is this guy here deliberate this one I think is another 1.99 gigabytes but these are pruned ones you don't lose very much with The Prudent ones they have less material but another base model SD 1.5 so we're going to be looking at SD 1.5 and we're also going to be looking at sdxl now to download comfy UI we come to GitHub and GitHub is full of all sorts of wonderful open source software we go to the project page and on the project page we have the installation instructions not very far down uh the the page there's some information about how it runs it's basically a craft nodes flowchart based interface and it's very easy to use once you actually learn the basics we have the installation instructions here and there's also a page called config UI examples which is a very useful place to check check out some of the some of the examples for for use cases for the software installing comfy UI this is a really useful place for Windows for Apple users for Linux it supports all those operating systems and it will also work with AMD and Intel graphics cards but the the main benefit comes from using Nvidia graphics cards especially an eight gigabyte Nvidia graphics cards as far as sdxl is concerned but you can run it on a CPU and when you download it and install it you'll have an option to run it on the CPU at least using the method that we're going to be looking at now the instructions for um Linux Apple you'll find them over here AMD gpus I think there may be an option to run yeah Intel Arc is down here Apple Mac silicon so all the instructions you need are here and there's also an alternative way of using an Nvidia graphics card now what we're going to be doing what I'm going to recommend and there's also an option for collab what we want to do is to download the direct link so once you click on this it will start start downloading go and get 7-Zip if you haven't got it and once you've got 7-Zip you just basically use 7-Zip to extract the zip file and the 7-Zip is open source it's free just follow the link and once you've done it's about one and a half gigabytes once you've finished uh downloading it extract it and then there are two files one file is going to be a GPU file for NVIDIA the other one is a CPU file so this is the option for those of us who have the Nvidia graphics card hopefully a powerful one for sdxl but if you want to use a CPU this is the option as well because you'll get a file to run it using the CPU you get a batch file that you can click on and it will install everything for the CPU it will run for the CPU or alternatively you can get it to run for the Nvidia graphics card as I say for the other ones just come here and the instructions are here and you'll be able to get up and running pretty quickly hopefully but uh for the Nvidia one literally just download the file double click on the on the batch file and it will install and run it's really that simple it's the simplest installation that I know for stable diffusion now the download for comfy UI is not that large I found it to be about one and a half gigabytes you get you extract it and you'll end up with this folder here the comfy UI Windows portable that is where everything is happening so we double click on that and then we can then choose either to run on the CPU or the GPU I would strongly recommend having a GPU running this the Nvidia GPU running this if you can the for AMD and Etc Intel you need to look at the instructions over at GitHub now once you start running the GPU you're going to get a window that comes up and I'm going to see if I can just locate that window it's going to look a little bit different on your system because I'm using Windows terminal which behaves a little bit differently but essentially what you have is a but command prompt that looks like this and the command prompt will basically need to be kept open once you're running comfy because I if you close it then it shuts down once you start running there are a number of things you'll need to do and I would actually do this first there's a folder called models and inside of models there's a folder called checkpoints you need to put your checkpoint files in there and the checkpoint files are the ones that we call ckpt files or safe tensors as you can see here these are all the safe tensors they're fairly large you're going to need 20 30 gigabytes of storage again that's not the minimum requirement that's more realistic probably you want maybe about 100 gigs of fast storage if you can and we put the refiner and the base in here there are a whole bunch of other models that you'll find in here so if you need to install other modules you'll find that there's a place for them here the important one is the vae one and this is where you put the sdxl vae the updated one and once you're ready to once you've done that you could then go ahead and read the instructions on editing this file which is called extra model paths yaml um you create another version of it with the example taken off and it will be just don't yaml that's a file that can be accessed by Python and that file could contain now we could take a look at it I suppose that file could contain instructions it could contain instructions that you could use for understanding where your files are stored making sure that a stable diffusion comfy UI understands where your checkpoints are stored so I've edited it to include storage for my checkpoints also my path for automatic 1111 Etc so it's a useful file to edit and the instructions are over GitHub let's get rid of this but do this stuff before you go ahead and run Nvidia bat or the run cpu.bat those won't run properly unless you've got a checkpoint installed and if you've got a checkpoint installed make sure you also have the vae because we're going to need that now the next thing I want to do is to actually show you config UI in action so this is the version this is the setup the workflow for comfy UI that I teased a little bit earlier and this is a very Advanced one it just um it contains a lot of stuff which just demonstrates the power of the software but uh before we get into that how does this come up I Believe come for you I will automatically open up a web page uh with so you you've got a server and the thing here is the server basically is acting as your server and if everything's going well with this guy the client which is the website or the web page is going to be running well as well so it will give you a an HTTP and you've got to just copy that Ctrl C or it will open up automatically in your favorite browser and let's come here and you can go ahead and paste that and you can see we don't need to do that because it's already open so what's next let me show you this workflow this is a complicated workflow and what I've been doing here is creating lots and lots of images now if you have a decent GPU it doesn't take a huge amount of time to create a single GPU but remember we've got this workflow this uh Ensemble of experts sequence so you've got the the base and then the refiner and it can add up if you're doing a lot of files if you're doing a lot of renders this is an insane number of renders that I did here we've got one girl here which is looking kind of fine and we can scroll scroll through and just see these other girls so when I ask the software to do was to just randomly uh randomly create a girl and once it's randomly created just continue to process and I can see each one that is actually created here and we're moving from the base uh the base renders to the refined renders now you probably can't see any difference because um you know I really need to zoom in to be able to see the difference myself but this is a really oh this is a really useful this is a really useful workflow when I'm trying to compare different outputs because sometimes the software makes all sorts of horrible mistakes and you kind of you need to be aware of how things are progressing so normally under normal circumstances this would be the final output for this one what I've decided to do is to sharpen the images so that we have an extra phase of processing and I can look at the original bass render and if I just hold down space and just move over I can see the final render and it just allows me to see where we started and where we finished I really like this this setup and this is the type of thing we're talking about on the on the course this is a little bit beyond the scope for for this video but you can see hopefully just how powerful the software is now there's also a history for a History Section that allows me to go through the the the the history and see what we created before so this is something we created a little bit earlier just random variations some with glasses some without glasses uh different uh different skin types and so on and just in case you can't see what I'm talking about the difference is I decided to create this which is it's got some special effects so hopefully if you can't tell the difference between the other ones you'll be able to you'll be able to see this one and see that yeah it actually is being processed so this is something that I did several times and you can tell that um it's possible this way to judge what's happening with the different renders so if we go to this one for instance I could maybe come up with a specific render see what happens when we put in a refinement and then do a bit more refining and then see what happens after a bit more refinement sometimes you find the more you refine you it's not seeing problems with this one I decided to give some more special effects so that you can see um the the effects of um of you can see that there is actually some kind of difference going on it's not just me talking about the the software doing stuff that you can't actually see anything about foreign so this kind of arrangement allows me to experiment with different outputs and different strategies and to see you know how all the different models work together so that's the kind of power of the software but what we can do at this stage is probably just clear everything and start from scratch so to clear everything just go and hit clear and boom we're done now the easiest thing to do typically is to load the default and gives you a suggestion do you want to load it yes or no and what we have here is the standard default this is what you'll be using if you're using 1.5 stable diffusion 1.5 so if you've done the the basics you will have installed dreamshaper 7 or 8 whichever one you want to install inside of the models checkpoint directory inside of that folder and we can choose that one so we can come to the load checkpoint and choose the the checkpoint you could if you wanted to choose the one for I think it's Runway is it Runway have model 1.5 yeah model 1.5 so you could use this one and I almost never used the one from Runway the official one I almost always use um dream shaper or one of the ones that we download from civit AI now here we've got the clip and code clip text and code we've got another prompt we've got the empty latent image the case sampler the vaed code and then save image and if I hit Q prompt you'll see what happens [Music] that was quick so what we've done here is created The Prompt inside of the save image and because this is a safe image node it saved this image it's actually it's not just a preview it actually saves the image so what happens every time we do this because the software is basically using these instructions or these prompts to act on the seed which is basically random noise and that random noise is then being acted on by the sampler to create an image of this size that image is once created is then decoded and then it's sent here so this is really the power of comfy UI you can really see just how um how how everything runs together if we hit the queue prompt again notice that we get that little outline around the thing that is actually actually active so this node here starts off being active then we get this node here and you get some really really interesting kind of surreal effects with this particular prompt it's a really good prompt this prompt comes just with the software it's it's inside the software um and uh it's a nice one now if you change the the the the the checkpoint the images that you'll get will change to match the the checkpoint the the training of the checkpoint uh the the vae as we can see here is actually um if we click on here and just watch there as I click on here you can actually see the pathway now the pathway is hidden by this text box but you can see the pathway so the vaed code is actually decoding using information in the checkpoint so the checkpoint is really important because it's linking to the vae which is decoding the image and giving us the image it's also connecting to the clip which is given the text the positive prompt and stuff that we don't want in the negative that this is the negative prompt [Music] thank you so you can edit it like that just right click title and positive and don't forget to hit enter otherwise nothing saves I've done that a few times now um the this is a very simple flow and we could if we wanted to because this is five one two by five one two we could come here maybe just open that up a little bit so that it's a little bit more visible so you've got all the space to work with now how am I moving around I'm clicking and dragging and if you click on the area with the grid and just click and drag it will click it will move everything it will move the entire workspace you can move it up you can move it down or it's either a property limits that you can how far you can go you can also zoom in and zoom out and you do that by scrolling so as you scroll in you go in hard and then you can come out all the way out here if you're working with very complex models very complex workflows you can sometimes have to scroll down quite a lot the low checkpoint is the starting point the clip has to be included next because you want to instruct the sampler the sampler then takes the input from here and it Crea it it creates the latent image and then works on the latent image to produce the output the output cannot just go straight from here to there because it's in that sort of weird language that weird mathematics that the sound that stable diffusion is using now this is the simple 1.5 workflow and probably a lot of people will be well aware of this particular workflow and that's because this is the workflow that we use in automatic 11 11. so if we continue to hit Q prompt what will happen is that we get a queue size and it will finish rendering and it will only ever run from the place where the last thing changed and you might ask what's changing here well what's oh that's beautiful it's such a nice little prompt what's changing here is the control r after generate so after we generate this what happens next what happens after we generate you could click here and it will decrement so the seed will decrement it will increment or it can stay fixed so if we stay fixed nothing changes except it runs because it wasn't randomized and it already randomized as soon as I hit Q prompt the first thing that it does is to randomize the number so because I randomized it had already changed the the seed if I hit it again because I've changed it to fixed it doesn't do anything because it's already fixed so first thing it does is to look here say should I change this or should I not if it's fixed it's not going to change if it's randomized first thing it does is update this number here so if you hit it with any of the other settings it will always go and run but only if it changed already because it wasn't fixed I had to click once I had to click twice and it did not change the first time because it it followed that fixed instruction so that's something that can catch you that can trip you up a little bit because the logic is not that obvious from from just looking at it now what we could do is just go and change to the I think Runway of 1.5 model so you can change the names of the the models you can give them names that make sense to you now because we changed the model it's actually going through a second time and you can see why people don't like the the the official models the runway model the official models 1.5 1.4 it's just not as beautiful as the stuff that we get from third parties so that's the reason I recommended going to Civic Ai and just basically uh downloading a couple deliberate it's a great starting point dream shaper and uh then using those I I honestly have almost never used the the official ones 1.5 and 1.4 uh dream shaper is almost always very good so when it changes because you've got randomized or increment whatever how do you find your original image you go to the history click on the history and you'll find the the image there and it will also bring up the the correct seed so I know the seed here produces this guy here if I hit you can see that it produced that particular seed that particular image again so you can actually find the seed by looking at the history and the history is useful for that is useful for it's a very useful place to be when you turn off comfy UI when this shuts down you lose the history but if you've got a save image the the image is all saved and you can actually go back and find those images again and I think we'll cover how to get the images again fairly fairly soon so we've basically got um an understanding of how this thing runs hopefully um what we'll do is once again clear and we'll recreate this from scratch so I'm going to come here I'm going to double click uh it brings up a whole bunch of options and we can search for checkpoint or load so we're going to choose checkpoint loader simple there we have it then the next thing is to just connect these up so we can connect this up like that and it says what do you want I actually want a case sampler because we were using a case sampler so we'll put that there maybe there we also want a clip so we want two clips so let's drag that out we want clip texting code that gives us our clip and then we want a negative prompt so we choose clip texting code again so that's going to be the positive prompt [Music] beautiful galaxy and the negative prompt I'm gonna put in nude foreign [Music] and we don't want text we don't want any watermarks if I try to cue prompt we'll get prompt has no outputs so it's figured out there's a problem somewhere and we need to fix that problem first let's go ahead and complete this we've got the model connected we've got code connected what about the vae now the vae goes to the vae decode doesn't it so if we click there and just drag we get an option and we can choose vaed code remember we've got to get the image out and that's going to be decoded by the vae vae stands for variable autoencoder it's a very important model that works it's a very important artificial intelligence that works with with stable diffusion we can then drag out an image and because I don't want to save the image I'm just going to choose preview image [Music] so to move these you just basically move the just click and drag um you can resize them as I did before but I think it will resize to it to a default image size now we also need to figure out what uh what what comes here because there's clearly a couple of things there that that are not connected if I move it here it'll be easier to see we haven't got the positive and the negative so let's take the positive and the negative and just connect those two [Music] and then we need the latent image now the easiest thing to do here is to drag out from here so we can drag out this guy and then say okay we want an empty latent image so we want an image of 512 by 512 it creates that by default so we should be able to now run the Q prompt and it should produce an output but it gives me a warning and this is nice the way it gives you These Warnings because you can kind of understand what's happening from the from the text but you're also getting feedback within the the window within the window here so you you get that same kind of message showing here so here it's telling me uh the prompt outputs failed validation the AED code required input is missing samples close so it requires another uh input to the vaed code if we run that again you'll see that message prompt outputs failed validation vaed code so we know it's the vaed code and we also notice the vae decode because there's a huge ring around it it really does tell you where the problem is required input is missing samples so it does give you a really good understanding of what's Happening both with this kind of big impossible to miss ring around everything and then a ring around this as well you can't really miss it can you you can't so we could just drag this out and that's not going to be a very good idea we need to find where we need to connect this and the only really meaningful place to connect it is with this guy here so we're going to take the latent from the case sampler and we're going to connect it to the samples so it kind of makes sense there's a flow to it the case sampler produces the latent and it goes to the samples and there's a big red oh there's the indication that that something is wrong with that you can't miss it you really can't miss it [Music] now it's using uh okay it's using a different checkpoint so if I change the checkpoint it will start from here because that's where the change is taking place uh let me see good there is a beautiful galaxy I I can't remember what the original prompt was let's put it in a bottle foreign beautiful scenery coupon so we start off with the checkpoint it we can see it flowing and it produces this beautiful image so dream shape is seven definitely a good one to start off with but remember the nude The nude is a really important one with some of these not safe for work ones so what we've got is a really nice image but we want to save it how do we save it we could drag out another option and choose save image and from some of these nodes you can actually do more than one thing so what I actually want to do is to recreate this one but we want to save it we can right click and choose save image that's one way of saving it or right click open image and then right click save image but instead I'm gonna go back to the history so we can demonstrate the save image command or the save image node [Music] now whenever you make a change it will take you back to the workflow so with each load you go back to the workflow whatever thing whatever whatever situation you had you go back to that so if you want to make changes load first and then make the changes and if you can remember to do that always you're better you're a better guy than me so we'll drag this out choose save image and uh just go there and we should see two of this now we run this prompt there we are so it allows you to change the the prefix but I usually keep it as comfy UI so we've recreated the original uh the original space that we get with default and we've added something to it we've added the preview if we wanted to we could hit Ctrl m and that will disable that so I wouldn't save this image every single time that I'm running the the prompt and I think if we maybe just cover the case sampler you'll have a good idea about how how everything actually knits together so with a case sampler it's the most complex part of the equation isn't it it's it's the only really detailed part of the the the what we're working with uh the seed gives us the noise you can change it by going up or down you can randomize it you can increase it or keep it fixed the number of steps is how many times the sampler works so a few steps usually is better than many steps but that's because it saves you time if you need to go up in the number of steps that's fine you just go up in the number of steps you run again and you'll see what happens might be a good idea for us to go back in history and then run that exact same prompt but this time keep it fixed and change the value to 40. what happens when you change the number of samples now one thing you'll notice is that it takes longer so for each sample for every sample same amount of time and the result isn't always better so I felt that the original was better but typically if you're struggling just change the the steps that can give you a different result the CFG is how much you want the sampler to refer to the positive and negative prompts uh how much you want the clip guidance uh to to be adhered to the higher the number the more it will adhere to this and if you want to know what the highest number is a simple technique is just to type in 10 000 or 90 000 or whatever just a really large number um then hit OK and then just click one time and it will just default to the highest permissible number now you probably want to keep this down at about 15 or less the sample name gives us a bunch of options of different mathematical models that we can use to run the sample and since the steps gives you the total amount of time that it's going to take to run the process changing this here can be a way of actually reducing the amount of time because different Samplers will run the iterations at different rates so some of them DPM fast you can imagine that one's a fast one Euler is a very good one but it's not neither fast nor slow and you can choose whichever one works best with your prompt and also which gives you the fastest result sometimes people use these Samplers based on the the output that they give them but I I find sometimes I just want the fast one because I can do more if I have a fast one the scheduler is a little bit beyond the scope of this video and then the denoise value typically you want to keep that at one so that's basically the case sampler and it works it works in in in in that basic uh way for most of the time when we get to sdxl there will be a different type of sampler that we need to use now um when you freeze an option by Ctrl m you unfreeze it my control M so if I wanted to make this work again Ctrl M would be the one that that would unfreeze it but keep an eye out for any signs that a particular process has come to an end because you've disabled a node by hitting Ctrl M it will give you some kind of indication on the screen that has happened now we're going to be moving on to sdxl next and what I want to do is to just point out that the sdxl has just come out and it's going to be evolving fairly quickly so I'm going to be keeping this updated this course updated and if you want to just keep up with the developments um there's plenty of stuff here and I'll be adding more stuff about new stuff that's happening within sdxl but we also have some very useful resources that we can find over at the comfy UI website on on GitHub so if you go to the conf UI website and just scroll down you'll find comfy UI examples I don't know why the color changes all of a sudden but we've got here if we scroll down a little bit we've got sdxl and if we go there you'll find that there are some some useful instructions here the SDA sdsl the sdxo based checkpoint can be used like any other regular checkpoint in comfy UI which is true which is true and it mentions that the only opt the only important thing is that for optimal uh performance the resolution should be set to 1024 by 1024 or other resolutions with the same amount of pixels but a different aspect ratio for instance these two guys here so basically you want to get about a million pixels about a megapixel for using the the base with the refiner you can use this workflow so where's the workflow it's actually within the images and you can do this with a lot of images so I've already saved these images and I'm going to try to drag them into into config UI sometimes it works sometimes it doesn't work [Music] so this time it actually worked which is great so we now discuss sdxl so grab one of those two images there's a complex one and there's a simple one I'm not in entirely sure which one I'm using here I'm using the complex one so what you would find is that any any image from comfy UI will contain the workflow and on top of that if you're using automatic 11 11 you can try to drag in the image and it will because basically stable diffusion so it should try to recreate the the workflow from a automatic 11 11 at least the current version of automatic 1111 it should try to recreate the workflow you may need to fix a couple of things and we'll talk about that actually I'll talk about that right now because I think it's important for you to understand the the key things that change right now we've got the checkpoint loader and another one here and notice that we've got loading base sdxl and load in refiner so we've got different different loaders here and then we've got clip text code for for the refiner prompt and another one base prompt then we've got a vae decoder save image we already know this we've already seen that before and then we've got something new here we've got case sampler advanced so you could run this but before you run it let's try it and see what happens [Music] see it's working [Music] it's already gone to the sampler the first sampler there are now two samples it's a base one and there's a refiner one [Music] then we've got our purple Galaxy in a bottle now the I think the prompt is exactly the same but what's different here is that we've got notes so you can actually produce notes inside of config UI and they're provided quite complex quite a detailed note on how everything works so if you want to know more about this this is your good starting off point just read the notes we've got daytime Sky nature dark blue Galaxy bottle and this Arrangement is a reasonably good arrangement to be working with you can try different Arrangements for yourself just like I did with my initial with my initial demo demonstration but we have here a second prompt a refine a prompt and notice that the refiner prompt the base prompt is one thing the refiner prompt is slightly different and that's okay that's okay it seems to work out just fine then we've got more notes Here one thing to really pay attention to is the um the recommended uh aspect aspect ratios so with the S SD 1.5 you're kind of limited to uh 512 by 512 and if you decide to use 2.1 I think it goes up to 768 by 768 with this guy here you'll find quite often that 10 24 by 1024 works best and you get that number again in the in the in the empty in the empty latent let me see if I can find the empty later we're looking at it so the empty latent image is 1024 by 1024 change it to one of these different aspect ratios but experiment and see what works well for your composition alright so let's talk about what's different here ah I mean I could just recreate this from scratch which might might be a little bit better for understanding but um let's take a look at this and maybe just try to deconstruct it we've now got what are called sdxl sdxo base and we've got a a base which we can use to load the the base um safe tense or or the base model we've got here a refiner which we can use to reload the refiner and it's fairly straightforward when you uh when you install this if you if you've renamed your files you'll need to make sure that you're choosing them correctly because it just won't work otherwise now um when we come down here it will store the name that that you see here the one that it was saved with not the one that that's on your system so feel free to experiment with this you already know if you change things in screw up you can always go to the history and the history is going to stay awake as long as your windows is option is is awake the uh the the command prompt window one thing that's a little bit different is the step control so we have the base sampler and we have the more advanced one now the way this has been set up you can see absolutely everything so we can see the basic sampler that we were using before and slightly more complicated one and the one that's a little bit more complicated needs some explanation so where the workflow is now if I hit the queue prompt we can see what's happening we can see that we're starting off with the sampler here because it's got randomized selected and it will then move on straight to the advanced sampler and then the decoder and we've got this new beautiful bottle now as a rule they're suggesting the the suggestion I've heard is to try to keep the ratio where it is so about you know looking about 20 to 25 for the the steps here so the end steps basically refer to where the model is going to end so we start off at 20 and we end at 25. it's a little bit difficult it's a little bit difficult if you're not seeing it uh in your own screen but essentially when we start what happens is that we start off here at number zero and then we finish at 20 and then what happens is that this sample of the advanced sampler takes over it starts at 20 and finishes at 25 and that ratio of about 20 to 25 40 to 50. that ratio is one that they recommend that's the kind of recommended ratio for the the base number to the to the refiner number you can of course do your own thing and see what what happens there there are no hard and fast rules and certainly when we go to the course I'll be discussing all sorts of different options which sometimes produce good results so that's basically how this one works I think we can also drag in the simple version and see what that one looks like [Music] so this is a somewhat simpler version that they've got um it actually looks a bit more complicated but we've got the same base models uh refiner model and when you want to create a refiner model it's going to be it's probably a base model loader and refiner model loader when you want to create these double click the the simple one so we want to use the simple one that one otherwise you you get an option which can create all sorts of confusion so use the simple checkpoint loader that one has been used here and um it basically looks a different color because they've colored it differently we go to uh proper no if we go to Colors we can choose a different color so that is how you change the colors um when it comes to yeah there's one other significant difference when using uh when using this particular setup which is um what what they're doing here is not necessarily what I would do what I would do is double click and choose vae loader and I would make certain to choose the latest version of the sdxl vae so my one because the early one I'm not happy with I'm going to choose the second one the one that I just downloaded and instead of you connecting the vae to the model loader to the checkpoint model loader what you would then do is just drag these vaed codes get rid of them like that and then just drag out like that [Music] and because we've only got one decoder this should work and uh you we now have three you now have four starting points and the starting points are the loaders here the loaders for the checkpoints we've got the vae loader and I'll probably want to make sure that um if I add any other any other decoders um I make sure I want to make sure I uh connect them to the load vae that allows me to use the the latest vae and that one I don't think is going to give us different results um visibly but you you definitely there's definitely good reason to use the latest one rather than the earlier one and it will work like that it will work with this as an input so we'll end up with four inputs we'll end up with the empty latent image that's our first input we end up with the load vae went up with these two as well we're gonna have four inputs that are kind of like the starting point for our diffusion and we'll end up with uh this kind of result and I think that is exactly similar to what they had in the in the original uh website if you go to the website they've got something that looks almost exactly like this so that is basically how it all works there's a lot of instructions here so I'm not going to spend a lot of time on this because you can actually sort of educate yourself with this and actually on the website there are lots of other examples you can save these ones and then you can just drag them into the window into the sdxl window you can just drag those other images to the sdx sdxl window and they'll give you different workflows that you can maybe experiment with and try out but this is basically how we do it it's a very simple option to work with and I think the only thing left to do is just give you a basic understanding of this aspect of the workspace we've got the we've got some settings here that you can change if you want to we've got the history and you can toggle the history we can save a workspace and we can load a workspace when we save it it's saved as a Json file so if you're having difficulty with with the the images tracking images in you can save the images you can save the workspace as a Json file and you can then open it up using load as a Json as a Json file um mode to access the Json file and the the the Json file will give you the exact setup that you had sometimes with the images the you know you know the the randomize and fixed options they can be a little bit weird they can be they can be a little bit confusing now um that's basically how it all works and um you when you've got lots of cues running you can choose CQ and you can actually cancel some some things that are running so that's basically how it all works and if you have any problems you can actually just check the the window just check this window here and it will tell you sometimes what's going wrong and give you an indication of what you might um maybe something you might have missed it will uh log there so that's basically how we do it and I think if you just follow the instructions in this video you should be good to go if you need more help of course just hop on over to the course uh it's it's something which will give you a lot more firm understanding of what what's what but that's going to be it for this one guys um I will see you maybe in the next week [Music]

Channel: Pixovert

Views: 15,293

Keywords: beginner's guide to SDXL, beginner's guide to comfyui, beginner's guide to stable diffusion, beginner's guide to generative ai

Id: yamusLEyt4g

Length: 64min 3sec (3843 seconds)

Published: Mon Jul 31 2023