SDXL Lightning For Stable Diffusion ComfyUI - How To Use And What We Should Know

hello everyone so I want to talk about the sdxl lightning that is created by bite den and they have recently launched this 12 days ago but then I have not talked about this last week because I wanted to wait for it to become a little more stable as we all know AI is moving so fast and sometimes the latest model from hugging face can change in the first one or two days or even after a week like previously we had stable Cascade it was just up uploaded one or two days ago and then after a few days it changed they made some updates to the checkpoints to improve the comfy UI and ET for me I would just stay calm and not follow the tail too closely or fall for misinformation and crashes so sdxl lightning has a demo page here and again I will share this link in the video's description you guys can check it out it's very simple just do a text prompt here or a a natural language style text prompt and you are ready to go with this basically this uses very low sampling steps and very low CFG number settings for xdx image Generations in this video I will try these four steps okay so basically they have released twostep four-step and eight-step models individually that you can download as separate files and I will talk about that later they have also released the all-in-one checkpoint models for comy UI and they have the unet checkpoint models for diffuser additionally we have the Laura's files for diffuser and comy UI as I am using comy UI I will be using the all-in-one checkpoint models and the Lura files I will test these two together and I'm going to use four steps for this testing experiment because they are not going to put everything in one file here as you can see when we click on the files versions you see this one step twep four step and xstep well you can use the four-step files and set the sampling step to 8 and 20 and even more but then it will not perform that well because they are built to run for four steps which is the optimum sampling steps in this model and if you use xstep then you will use xstep for the best performance in step model files so let's try it here first of all if you need to download download these Laura's files and then the checkpoint files are good enough for comfy UI if you're using diffusers then you have to download the unet instead of the checkpoint models for this one it's as simple as that and once you download the checkpoint models again put them in your models checkpoint folders and for the Laura's files put them in the Laura's subfolders it's as simp simple as that just like what we normally do for checkpoints and Laura's files okay so I'm not going to go through how you save your files and drag and drop them into those folders because that is a very basic step that we have covered in many videos already let's take a look at that here I have the Laura models of the sdxl lightning I already have the four-step model which is similar to the LCM concept we previously used with LCM Laura in this case it's Laura for sdxl lightning so choose that instead of LCM and then we follow the steps only here set the sampling uh step to four and CFG to one now I have tested other numbers for CFG like three four and even up to eight but that will oversaturate the AI image so if you have four steps cfg1 is good enough even when I set it to CFG 2 the image becomes over saturated with color saturation like this one as you can see it instantly generates the image it's fast as it is here I have two Samplers the first one is set to 0 seconds and the second one is the second case sampler which I'm using to enhance the image that will be less than 2 Q that's how far we go how fast we can use sdxl lightning which is very similar to sdxl Turbo it's almost real time generated but compared to sdxl turbo it's simpler to connect we don't need extra custom nodes to generate images just using normal checkpoint nodes and Laura's note and you can generate an image with that the concept is very similar to LCM with very low sampling steps and CFG and you can run it without any problems you can use normal schedulers or use the sgm uniform which performs better for image generation using sdxl lightning I found that using e ancestral is the best one to perform so that's the setup so far let's try it with a new workflow diagram here let's clean it up and let's build one together so first we got to load the default workflow so this is the default workflow from comfy UI here now we are going to make a few modifications on the front end and you are able to use sdxl lightning if you only have checkpoint models you can download just the sdxl lightning checkpoint models if you want to use four steps you can use that you can also use two steps or eight steps depending on your preference but I prefer to choose the middle one right in the middle so I can test the average performance of sdxl lightning overall right here you can simply choose the checkpoint models of sdxl lightning and enable that so it's pretty easy to set up without any modifications on the front end here and it's totally different from sdxl Turbo when using sdxl turbo we had to use special custom nodes for connecting but here we don't need any of that so just set it to four and let you guys check it out I'm using the sgm uniform and we can take a look at this one the first time loading it may take a while to load up the checkpoint models which is normal but then as you can see the speed of generating images in the case sampler takes no seconds yeah just no seconds here so let's set up preview and track one more time here is the image very easy and let's make it larger even a large 1024 image is generated in less than one second so if I really use this model to generate images I will use two Samplers because the first one takes less than 1 second that way I can have more time to create another case sampler maybe I will do a latent up scale here for very basic custom notes or comfy UI workflow I can do some very basic stuff here and generate fast and again I don't need to go very complex doing all those crazy fancy stuff connecting 100 nodes here to create a good image that's how it's working and let's see right here we can do vae with this one actually it doesn't have vae so we need to load our own vae loader and load an sdxl vae that will perform better for the image I have my sdxl vae and that will be going here that will be the image for the second case sampler let's check it out so this is the first example and now we have the second one coming up well the second one is going to be slower because I took 20 steps let's see what happens with 20 steps right so with 20 steps it looks like this and as you can see I didn't set anything I just loaded the case sampler and let it run by itself then I upscale it two times the size of the original image so it's like two times larger than the original image everything here has more details and is more enhanced with richer colors compared to the previous image but it doesn't follow the same pattern because I forgot to add the noise here so let's set up the denoise to around 0.55 and again use for steps let's see the Speed and Performance of this again set the sampling scheduler the same way and let's try it out hopefully we'll get the same result here yes we got the same one as you can see we have more details here compared to the previous image it's not just a solid rock but we can see different rocks and trees here that is the basic process of using checkpoint models to load sdsl lightning now let's move on to another example using Laura to load sdxl lightning I prefer using this method if you want to achieve very fast image generation I prefer using sdxl models because once you have good models like juggernut XL and realis XL which are well-trained models with a wide range of sty to play around with you can simply improve the speed within a second by using Dragon odd XEL to create an image moreover this method is more flexible because I can use other Laura models for example I have Laura beautiful girl for Jordan 2 which is another Laura style for enhancing based on the checkpoint models after all I can pass these models to this workflow and that will yield better results yes the images will be better using the sdxl lightning Laura models and in the end we can lower the sampling steps to increase the speed of generation let's try four steps and one CFG we can use a futuristic text prompt for this example as you can see the first time loading may take some time due to different checkpoints and configurations however subsequent loadings will be incredibly fast even in the second case sampler it only took two seconds to generate the entire image this image has been Laten upscale two times using a latent image size of 12008 a 768 and this is the result of that upscaling so basically I can incorporate more Laura Styles here along with the checkpoint Styles and other additional Laura Styles before passing it to sdxl lightning this allows me to lower the sampling steps in the case sampler and it's my preferred method of using sdxl lightning rather than solely relying on the checkpoint models with this approach I have the flexibility to apply special fine-tuning to the checkpoint models or even use Laura models so the performance is the same all these images are generated in under 1 second when using 1K sampler so it's acceptable and suitable for my needs I'm not too concerned about the speed or performance especially since I have an Nvidia GPU 4090 so it's not a major issue for me however I wanted to test the speed and how fast it can generate a large siiz image with at least average quality I can't claim that these images are of excellent quality but they have a very good average quality considering the low sampling steps and low CFG use this is how you can achieve fast image generation using using sdxl lightning when compared to LCM and sdxl turbo this method is much simpler to set up just load the Laura model choose the sdxl lightning Laura option lower the steps and you're good to go you'll have lightning fast speed using sdxl I hope this inspires you to explore this approach it may not be suitable for all scenarios but it works well when you want to quickly generate an image to get Initial Ideas once you have a better base image you can enhance it using other Samplers or incorporate it into another workflow to further refine the details that's how those low sampling step fast loading generation models like sdxl lightning LCM or sdxl Turbo are used so one more thing I want to mention the animate diff workflow for this sdxl lightning I have try out with the workflow gen or some videos but it is not really performed good in detail tail and buried even I set up to step 10 and CFG 2 it is still blurry so I'm testing for more give it some time see if there's any Improvement or maybe using other way of connecting animations using sdxl lightning but I don't think it's good choice to use sdxl lightning for animation just the same concept as sdxl turbo so see you guys in the next videos have a nice day see you

Channel: Future Thinker @Benji

Views: 2,115

Keywords: SDxl Lightning, ByteDance, AI model, image generation, checkpoint models, Lora's files, four-step model, sampling steps, CFG number settings, AI technology, user-friendly interface, download, setup, image enhancement, connectivity, complexity, SDxl Turbo, Lora styles, high-quality images, stable diffusion, ai art, stable diffusion tutorial, stable diffusion prompts, stable diffusion ai, install stable diffusion tutorial, sdxl lightning

Id: PK54CJhftps

Length: 14min 1sec (841 seconds)

Published: Tue Mar 05 2024