End of Year ComfyUI Updates for Stable Diffusion

hey guys how are you doing today in this video I'm going to talk about confy UI updates confy UI got updated yesterday and uh we have quite a few updates we have SD turbo stable zero group nodes and more features related to Performance this is the official confi blog and as you can see the update was December 19th that was yesterday for me and the first update we have is about sdxl turbo we have the workflows those were there from before and you can read it from this link here what we have new in this update is that we have an SD 2.1 turbo model and this one is based on the SD 2.1 model you can download it from here as usual the model is roosted on hugging Pace click on the download button and save it in your conf UI checkpoints folder if you are not familiar with sdxl turbo I have a video here on the channel called how to use sdxl turbo INC comi plus I have another video that talks about a gradi application I made for sdxl too you can watch both and you will learn more about the sdxl tuu model SD kbo model is a distill version of stable diffusion 2.1 and it is train on Real Time synthesis so it's not going to give you the best result out of the box the community SD XL turbo model is better but hopefully the community takes this model and find unit to have amazing results let's take a look at an example now click on the sdxl turbo example page here this will open the confy UI examples click on the image drag it on top of your confy UI release and you will have the sdxl turbo workflow give it a quick refresh go into the load checkpoint and select the SD 2.1 turbo model you can leave everything else as default for sdxl Turbo because this one will use the same key sampler the same custom sample and we can click on Q promp so once the model is loaded it took 2 seconds to generate an image 512 x 512 on my machine now keep in mind that I am using a GTX 1650 and I have only 4 GB of RAM so if you're using a recent GPU then you're probably going to get sub one second generation with it however the quality is not the same as sdxl turbo as you can see the prompt here is the default one the basic a beautiful landscape with a fox inside the bottle but uh we did not get the fox inside the bottle we sort of got the fox here but um it's it's not really great if we do the same example with the same seed with sdxl turbo I'm going to duplicate the workflow and change the checkpoint to sdxl Turbo leave everything the same including the seed and click on generate so in my case the sdxl turbo took 4 seconds that's 2 seconds extra but now I have a better image I would say so this one is from sdxl Turbo and the top one is from this new model the SD turbo or SD 2.1 turbo and the issue I have with this model is that sdxl 2 2.1 pretty much all the 2.1 models they are trained to work with negative prompts and when we are making a turbo model the tuo model does not utilize the negative prop we can take an example here so I'm going to click on extra options click on Q and I will click on Q prompt this will make confy UI keep listening to changes in the workflow the moment we have a change it will register it and it will execute the promp so if I go into the negative p i put the comma you can see Q size one and it is generating an image here we got the same image I can put zombie here it's generating I get the same image no matter what I put here the SD turbo model will not take the negative PRP into consideration therefore we cannot really tell the model to remove things from the image we can only use the positive from to influence the image so I'm not sure how good SD 2.1 turbo is going to be since the base model that was used for it the stable diffusion 2.1 works best when we have the positive prom and then when we have a negative prompt together but it is an update hopefully the community can take it and find shun it to give us some amazing models next update is about the front end Improvement and this one is a big one we have group nodes in my last video I talked about how to make your own custom node here and in that video I show you how you can combine the positive and negative promp into one so I made this a positive promp is at the top the negative promp is at the bottom bottom using the custom NES now with the group NES update we no longer need to create our own custom n to combine NES into one we can simply go to config UI load any of your workflow let's say we want to recreate that positive and negative PRP into a single note just select the positive FR select the negative FR right click on a blank area on your canvas and then choose convert to group node give it a name I will say prompts click on okay and right away you have a node this is new node that contains the positive promp the negative promp and you can do this for any node so if I load through the default blow I can select everything right click on the canvas click on convert to group node and I can call it efficiency click on okay and now I have my own version of efficiency node so if you're familiar with the efficiency group of node basically compact everything into one and then you can generate images with it let me change the checkpoint model to sdxl Turbo and I can click on Q prompt and at the top here it will tell you how many NES we have in this and the position at which the progress is at right now so right now it's at zero it's basically loading the checkpoint we are at four five six this is probably the key sample it's taking a while we are at V code the preview image here and we have the image inside this one node you can resize you can do anything that you want with it pretty similar to how a traditional nude would work and previously I was doing this so I was grouping each section into its own group so I have a group for Ludo one for prompt sampler V and then output this was utilizing the groups but now we can simply grab everything and put it into one nde now in case you want to get back this node this new node that you've created let me clear the canvas can right C cck click on ADD node go to group nodes at the bottom click on workflow and then you will have all of your group nodes that you've created in my case I've created the prompts and then efficiency if I click on efficiency I have this efficiency node now the next update in the front end is about the undo and redo so if I go here let me load the default workflow and let's say by accident I've deleted the positive prom before this update I would have to go in right click add a new clip text and code and then connect everything with this update I can simply press control and Z on my keyboard or Zed and it will undo the previous step so that I have my positive prop back and let's say I do not want the positive promp or maybe I've deleted the positive promp negative promp and now I want to undo so I have the negative promp here I will undo once again press contrl Z but let's say I've changed my mind I do not want the positive promp again press control y to redo so it will go one step back okay so these are shortcuts they are standard shortcuts in pretty much all application but now we have it inside of confy UI and it works for the nudes as well not just text okay moving on we have one more frontend Improv movement and this is about primary noes let me load the default workflow again I will change step count to an input box click on it drag it down click on ADD n utils and I will choose primary now so if you are making a workflow and you may have heard about making buses you create a section in your workflow that will contain All The Primitives so this will be your input for step count the seed CFG all of that will be here then you will use the reroute to send this over to different workflows right so we have our Primitives and we send it to workflow here one workflow is here another is here another is here and you can you reuse the same input for each of these workflow however the main problem that we had before is that these primitive notes they could not go through a route but now we can there is one little things that you need to keep in mind when using the Primitive node and trying to reroute it so if you want to reroute the model for example or click on the model go here select re route and uh we have a model reroute node here can right click on it and should type by default this way we know that this is the model OKAY similarly I can do it for the clip it works like you would expect it to work however for the Primitive note if I click on the int drag it out and I select reroute it doesn't connect and I think this is not a bug but more of a feature limitation because this node here doesn't know what it's supposed to change into unless we have this line that is connecting it to the input field there so in this case we connect it to steps so it knows that it's supposed to change itself or move itself into an integer field but when we send it over to a reroute it doesn't know what it's supposed to convert into so to use this correctly you will need to select the input field send that input field to a reroute this way the reroute knows that this is an integer then we send the input from that route to the output of the Primitive in order to wire it correctly so it's just that little step that you have to work backwards in order to get the reroute to work for primitive I'm really happy about these three improvements the front end improvements is really good here next we have a new model called estable 0123 and this is an sd1 XX model it can generate images from different view so if you give it an object from the front view it will go ahead and generate the back view for it and as you can see here there's no positive or negative text input it's simply an image checkpoint that has a clip Vision embedded in it and it knows what this image is and it will just generate the back side of it I'm assuming if you give it the back side it will try to recreate the front side now I have the model downloaded here but uh for me when I click on Q promp I get an error that tells me that the header is too large I'm not sure what this error is saying right now if anyone of you know what this error is and how to fix it please let me know but based on the example this is what it's supposed to do next we have performance Improvement this is the fp8 support it's basically how P toch works with different format and we have two formats here the author recommends using the fp8 if you are running out of memory usually confy is really efficient when it comes to loading the checkpoint but just in case you're having trouble and you want to try you can enter these values into the batch file so let me show you I'm going to copy these two values here one is for the clip and one is for the model so the one that that says text and code is going to be for the clip and then the one that says unit is for the model you need both now I am assuming most of you is using the portable version of comy one then you will have your run Nvidia GPU WB or CPU you need to right click on the batch file whichever one you're using go into show more options then click on edit this will open it in new pad and then at the end is where you're going to paste those arguments the fp8 for the model and then the text encode then save it and try it out now for those of you who want to experiment that is python 3.12 as well as torch or I should say py torch 2.3 that got releases and you can update to this python version or py torch version and then see if you have any improvements you're mostly going to look for Generation speed now next we have the self attention guidance and this one is still in the testing category I'm going to use the default workflow here but you can use any workflow for testing and the way that the self attention guidance work is that it's supposed to make images sharper and more consistent however it's at the cost of a slower generation so right click on an empty canvas click on ADD node and you will go into the for testing section or category then select the self attention guidance from there and this one is very simple it takes a model and it outputs a model so for testing purposes I'm going to by pass the self attention guidance change my seed to fix and I'm going to put a closeup portray of a girl this model is mainly trained on anime characters and then in negative I will put some save C text and click on generate so right now I'm not using the self attention guidance node it's the traditional node it's going from model directly into key sample okay it's done and uh this is the image that we got I'm going to add a load image node here I would right click on the output image click on copy clip space and then go into the load image right click and click on paste clip space this will save a copy of this output then I would enable the self attention guidance and click on Q promp so this time I'm going from model so loot checkpoint to self attention guidance and this new model is getting sent to the key sampler so it's completed on the left we have the self attention guidance one and on the right we have without the self attention guidance one and there's very little differences between these two so if I open both of them um you can see this is the without and with without with and yes there's a change but uh I'm not I'm not quite sure about whether it's making it better or not the they both look the same to me personally and this is odd so it's subjective everyone has their own opinion let me know what you think about it if it is improving the image or it's not really worth it in terms of generation Speed without the self attention gu guidance it took me 40 seconds to generate an image but with the self attention guidance it took 1 minute 11 seconds to generate now this is experimental but I will personally not be using it it's it's almost 30 seconds of extra time for me just for little change now next we have the prep negative and again this is also experimental in order to use it right right click add node go into for testing and you will have the prep negative at the bottom now this one is a little bit weird because you need to have an empty conditioning sent over to the empty conditioning field then you take the model send it to the model and this model the output goes into the casy sample I'm going to remove the self guidance one and uh let's generate so it's supposed to make the images more consistent by making the negative prop have more precise effect and again it's going to be at the cost of a slower generation time I have the same seed here and we'll see if it's going to make the images better or not it's completed on the left side we have the prep negative nude and on the right side it's the image without the nude and uh it's a totally different image so it's taking the negative from that we have here and it's it's making the negative section more prominent and um we can see here it's a totally different image but in my opinion this left side is better than the right side the left side is with the prep negative node but this is just one generation and if we look at the terminal it took 1 minute 6 seconds for me to generate that image while without it it was 39 seconds so that's about 25 seconds extra in order to get this image again this is experimental and it's completely subjective use it experiment with it and let me know if these two nodes are worth it next we have a new model called segmine biger model and this is supposed to be used similar to your sdxl base checkpoint so I have a basic workflow here where I have the sdxl segmine Vega as my checkpoint then the update also suggest to use an LCM Lura in order to get images faster so we can use a l CFG and a step count I'm using the LCM Lowa sdxl for sdxl also the model is an sdxl model then I'm going into the model sampling discrete using the sampling method LCM using a positive a negative I'm using five steps and a cfj of 1.8 the sampler name is also LCM now once I'm generated this is the image that I got and it's um it's not bad when I change the prom to a character this is what I got now the same promp with sdxl would have given me better result with a base model plus the refiner but for me it would take me about 8 to 12 minutes to generate an image this one took me about a minute but the quality of the image is not great so again experiment with the model and let me know if uh you got better result now next we have a save and animated PNG node so if you've been doing animation most likely you saving your animation as webd but now we have a new nde that's called save animated PNG that can save in PNG format so it's going to be like um GIF or GIF where the images will go one after the other and here's an example I was trying it with a couple of images and as you can see it's just take all the images depending on the frame rate it's going to do this or this this one is from a slower frame rate I have only four frames as you can see at the bottom there are four images and this one is four frames but with a higher frame rate now of course these images are not to be used to make some kind of video but if you have consistent images then you can make sort of video now next we have performance update so we have a deterministic that can be used and we have a GPU only that you can use again these are arguments and the way that you'll use it is by going into your batch file click on edit and then you will add these arguments here and this was the final update on the comfyi blog for this J we've got the SD 2.1 turbo the front end Improvement I think is for me personally the most valuable the group notes really amazing rewriting the Primitive nodes again really good and then functional it me control Z and control y for undo and redo is really good as well the different models that regard subjective as well as the testing the experimentation for self attention guidance and the prep Negative they they do come at a cost of slower generation time but the prep negative gave me a better result the self attention guidance for me the difference is barely visible and some minor changes as well as they save animated PNG in case you're doing animation and you want to have a PNG version instead of a MP4 or webp format so that was it for the update thank you for watching until the very end let me know how you feel about these updates I would like to say a big thank you to all of you guys so the channel has reached 2.0k so that's about 2,000 views and the watch time is about 114 hours right now and this is really amazing reaching the end of year I never expected to reach these num and also a big thank you to all of you who subscribe currently at 95 subscribers so hopefully by end of year the channel can reach 100 subscribers that's going to be huge for me big thank you really appreciate your support guys I will see you in the next one

Channel: Code Crafters Corner

Views: 7,258

Keywords: comfyui updates, comfyui, stable diffusion, sd 2.1 turbo, group nodes, undo redo, reroute primitives, self-attention guidance, prep negative, view synthesis, segmentation, save animated png, performance

Id: OXvKtRwxZNs

Length: 26min 23sec (1583 seconds)

Published: Wed Dec 20 2023