SDXL Refiner Extension: simplify the image generation process!

hello everyone last week we saw together how to use the SD Excel model within automatic 1111 using the text to image and image to image tabs today I want to do the same by using the refiner extension so we are going to see how to download it and how to use it it's very simple so it's going to be a very quick tutorial but hopefully helpful the first thing we want to do is to initialize automatic 1111 so this is the main interface of stable diffusion and I assume that you know how to or you have already downloaded the new models from hugging face if you haven't you can watch my video on how to do it and the first thing we want to do is to download the extension as any other extension on automatic 1111 we need to go inside the extensions tab we go into available and then we click on this load from button thank you and then I have it installed already so I'm going to untick this but in your case if you don't have it you can control F and look for refiner and this is the extension we want to install now again in my case I have installed it so I have installed here but in your case there will be the install button you just need to click on that once this has been installed you need to go back into installed and then you will see the new extension appearing in here exactly here as the web UI refiner make sure that this is ticked and then apply and restart UI something very useful you probably know about this but it's good to mention it if you want to know a little bit more about the extensions you can click on the link next to the extension like in our case we can click on here and then the main GitHub page will appear and there are some information about the extension who developed and and how it worked and usually also how you have to install it so in this case we don't have many information but it's telling us that the extension loads from refiner checkpoint only unit and replaces base unit with it at last steps of generation for who is not familiar on how neural networks maybe this is not very easy to understand but for who is familiar with that probably it's quite useful anyway we'll understand this step by step today let's go back into stable diffusion so again apply and restart and we'll see in the text to image towards the bottom we have this very finer if we press on the little arrow you will see that you can enable it you can choose the model and then you can decide how many steps you want to use the refiner for you have this extension in text to image and image to image as well so if you scroll down again you have it in here how the SD Excel model works so it's composed by two model you have the base which is Standalone so you can use it by itself and then you have the refiner the refiner is an additional part as the name says it's a refinement state for improving the image generated by the base model so you cannot use it by itself we need to use it together with the base model so what we were doing before we were generating the image in the text to image tab using the base model and then we were moving that image inside the image to image tab swapping the model to the refiner and improving the image generated with the base model through image to image what we are going to do today with the refiner we are just gonna use one tab which is the text to image so we don't have this swapping tabs which is great we are going to use different percentage of steps for the refiner so we are going to see different results together and then we are going to compare the results using the previous methodology so swapping from text to image to image to image just to see the difference I don't have much creativity today so I'm going to use a picture from realistic vision on cvti so I I picked this uh this one I'll copy generation data I'm going back onto my stable diffusion and I'll paste the data then I'll press this button for moving all of the settings across my text to image tab I can keep the seed or I can maybe use a random seed and then I'm going to change a few things because this picture was generated with this generation data for stable diffusion 1.5 but we are using St Excel so I'm going to increase the sample steps to 30. I'm going to use 1024 I would like to have a landscape image so maybe I will reduce this a little bit like that I'm going to remove these two loras because they're not gonna work then I think I'm quite happy with the with the rest Okay cool so for now I'm not touching this refiner section and I'm just gonna press generate obviously the model I'm using as base model is the sdxl based and this is the picture I got with the base model only now we want to prove this picture right so what we can do we are going down into the refiner we can enable refiner I'm going to use the same seed because I want to use exactly the same image and see how the refiner will improve my image so I can compare the results I'm going to use here model refiner 1.0 and then I'm going to keep the default value which is 20. what is this 20. this is the percentage of refiner steps from Total sampling steps let's make things easier actually so let's use 100 something steps yeah so let's assume that we are using 100 simply steps so when we use 20 it means that for the first 80 sampling steps stable diffusion is using the unit from the base model and for the last 20 steps which is the 20 of 100 right it's going to use it's going to replace the unit with the unit with the structure of the refiner model so with 100 was easier right okay I'm going back to 30 now sampling steps and I'm going to keep 20 20 for the refinery is what is recommended by who created this extension and it's even written here it's recommended to keep the percentage at 20 percent because higher values may result in distortions and as I said before the refiner cannot work alone so you cannot increase the percentage 200 percent to use only the refiner but if you know this if you want to scroll this up it will stop at 15. cool so let's start with 20 percent in here and I'm Gonna Keep 30 sampling steps so we can compare the the two pictures and I'm gonna press generate or before pressing generate you need to enable the refiner don't forget it and this is the image we generated using the refiner now we can compare the two results on the left you have the image generated using only database model while on the right you have the one generated using the refiner as well so it should be more detailed ideally so now keeping the sampling step to 30 I'm going to change my refiner steps to 10 30 40 and 50 percent I'm not gonna show you all of the processes because it will be too long so I'm just gonna show you my result which are this one the outputs all look good but obviously I have a preference which may be different from yours and I think it's this one now the results can also change depending on the number of template steps we are using overall so in my case I'm using 30. what I'm going to do now for testing it I'm going to increase the number of sampling steps and I'm going to do the same exercise changing the refiner percentage steps and this is the result so as you can see you can achieve different results using different sampling steps number different percentage of refiner they all of them they look great and in this tutorial I would like you to have an idea on how many sampling steps you may want to use and how much percentage of the refiner you want to use obviously is is good if you test it by yourself because depending on the input image on the image you're generating on the different descriptions and also different you know sampling method you're using you can achieve different results now I just want to compare these results to using the base model and the image to image tab I'm going into image to image tab I'm going to upload the image I generated using only the base model which is this one what I can actually do I can go into PNG info I can upload the image and then I can send to image to image so we have all set fold here then I'm gonna go down and I'm gonna change the denoting strand so I just want to compare what's happening here so you know what actually I'm gonna use the plot for doing it x y z plot and I'm gonna use different denoting strand so what I would generally use for the refinery it's a value between 0.1 and 0.3 I want to try more values in this case so I will go for 0.1 0.2 0.3 0.4 0.5 0.6 0.7 the higher the noise the more creative will be the result the lower the less detail the image would be compared to the base model so there is a trade-off right but that's why I was saying that the best value is the recommended ones are between 0.10.3 depending on how much you want to change the base picture I'm going to generate oh I forgot something really important sorry I need to swap the model to refiner okay and then I'll press generate okay this is done so as you can see the higher the noise in strand the more different is the picture but it's still nice it's more detailed sometimes maybe too much detailed yeah I looked great let's now compare these results with the previous ones so I will say that obviously you can use both of them they're both good using the refinery extensions it's easy and quicker because you don't have to swap from one top to another with the image to image tab is not as quick obviously but I feel like you can push more for the details although sometimes pushing too much with The denoising Strand is not very good again it depends on what you want so yeah that's it I hope this was useful thank you very much for watching and see you in the next video bye

Channel: Laura Carnevali

Views: 8,331

Keywords: stable diffusion, diffusion, ai art, diffusion model, generative ai, generative art, stability ai, ai artist, imagen, nft, install stable diffusion, install stable diffusion on mac, install stable diffusion apple silicon, apple silicon, stable diffusion on m1, stable diffusion with python, stable diffusion hugging face, stable diffusion github, stable diffusion v1.5, stable diffusion tutorial, stable diffusion xl, refiner, refinement stage, new sd, sdxl 1.0, sdxl, sdxl base, v1

Id: 5F2z1FknAfo

Length: 11min 47sec (707 seconds)

Published: Thu Aug 10 2023