Stable Diffusion Tutorial: Install and Use SDXL 1.0 Base and Refiner Models with Troubleshoot Tips!

[Music] hello everyone keyboard Alchemist here and welcome back to another stable diffusion tutorial it's an exciting day today because the highly anticipated sdxl 1.0 model from stability AI was officially released just a few days ago and it is now available to use with automatic 1111. in case you don't know already sdxl 1.0 is the latest and greatest open model from stability AI that is boasting a whopping 1024 by 1024 base image resolution and Promises to deliver text to image generation that dwarfs its predecessors I got my hands on it as quickly as I could and started testing it out and let me tell you the results did not disappoint the best part about sdxl is that you have the option of running it locally on your PC with automatic 1111 for free as long as you have a graphics card with agbs of vram or more [Music] in this tutorial I'm going to show you how to download and install the sdxl 1.0 base model and refiner models I will also show you how to update your existing automatic 1111 web UI and show you some troubleshooting tips if you need a tutorial on a fresh installation of automatic 1111 click the link on the upper right hand corner or in the descriptions to view my previous video that provides a step-by-step guide on how to complete a fresh install finally I will show you the results of my initial tests with the sdxl base model comparing images generated by some of the most popular stable diffusion version 1.5 models with SD XL so without further Ado let's get started to start let's go to hugging face and download both the sdxl base model and the sdxl refiner model you will need to go to files and versions then click on the down arrow icon next to the safe tensor file then go to the refiner model hugging face page and do the same thing for convenience I will leave links to the hugging face pages in the video descriptions when the downloads are complete go to your automatic 1111 Main folder then the model subfolder then stable the fusion subfolder and drag and drop the safe tensor files here once the files are copied over you can delete the other copies in your download folder if you are like me and have been avoiding or didn't bother updating your automatic 1111 installation then you will most likely have to do it now because only the latest automatic 1111 web y supports sdxl but with that said I suggest you try loading sdxl in your web UI first if you could load the model weights then you don't need to update and feel free to Skip to My testing results portion of the video but if you cannot load the sdxl model you will likely get an error like what I got here then you will need to follow the next steps to update your current installation you need to navigate to your main folder click on the top navigation bar and type in CMD then the command prompt window will open in this window type git pull the get pull command will try to pull updates from the main automatic 1111 development branch you might run into a message like what I have here it says there is no tracking information for the current Branch please specify which branch you want to merge with we want to merge with the master Branch because that is the main branch containing the latest released version of the web UI copy the command that is shown on the screen here and modify it like this and click enter you should see a message that says Branch Master setup to track origin master this means you are now correctly set up to merge with the master Branch then type git pull and hit enter this should allow you to correctly update to the latest version of automatic 1111 now you can close this command window and go back to your main folder go to your web UI user.bat file and double click on it now you see that message at the top saying cloning stable diffusion Excel into your local Drive that means the installation is happening correctly so just wait for it to finish it will only take a couple of minutes once you get into your web UI on the bottom of the page you should see that the version is 1.5.1 now we can try to load the sdxl base model again it might take a few seconds but eventually you will get a message saying that weights are loaded let's do a test image generation here I'm only using a simple prompt to test out whether the sdxl model and web UI is functioning properly we will generate better images later just a fair warning since the sdxl model is fairly vram intensive the image generation will be slower than if you are generating images using say version 1.5 models if you have more vram the image generation will be faster generating the initial image took me more than two minutes using my Nvidia 3060 TI graphics card with 8GB of vram that is with xformers on let me know in the comments below if you had similar experience using sdxl and feel free to share any settings or tips that might make image generation a bit faster in sdxl note the first time that I tried to generate using sdxl it alerted me to add the dash dash no half VA flag to my command line arguments I'll show you in a minute how to do this the sdxl model will need this command line argument to work properly in order to add the dash dash no half vae flag go to your web UI user.bat file right click on it to open it with notepad plus plus as you can see I have many command line arguments set up here I will add notes regarding each command line argument in the video descriptions you will want to add the dash dash no half vae flag in front of xformers don't forget to save the file close the previous command window then restart your web UI congratulations you have just successfully updated your automatic 1111 web UI and installed sdxl now we are ready for the fun part before we start do me a favor and click the like And subscribe buttons to help support this channel your likes and subscriptions Help Me Grow this Channel and allow me to continue making quality content thank you okay so I wanted to see how images generated with sdxl stack up against fine-tuned version 1.5 models I picked three very popular version 1.5 models they are dreamshaper 7 minor mix version 11 and realistic Vision version 5.0 in case you don't already know the fine-tuned models are good at many things but each model is specialized in some way dreamshaper is a model that excels at generating illustration style images minamix is a great anime model and realistic vision is good at generating photorealistic images I set up 12 different prompts and saved them as styles styles are simply saved positive and or negative prompts and you can reuse them by selecting them from the Styles menu on the right hand side of the web UI for all 12 prompts I made them very simple with as few words as possible because I want to give the AI some freedom and see what they can generate on stability ai's website they say that sdxl only requires a few words to create complex detailed and aesthetically pleasing images so I figured let's put that to the test by using as few words as possible in the prompts I will link the Styles or prompts that I used in the descriptions for reference I set up the script section to generated X Y plot with the different models on the x-axis and styles on the y-axis here are the settings this is a 4x12 grid with a total of 48 images if you want to learn about how to generate XY plots or XYZ plots I have a great tutorial for this feel free to check out that video by clicking on the link in the upper right hand corner or in the descriptions I am generating all images using the Euler sampling method as this is an all-purpose sampler that I know would work well with most models sampling steps set to 40 and CFG scale set to 7 which are basically middle of the road safe settings then width and height set to 1024 by 1024 because sdxl was trained on 10 24 by 1024 images and this will show us what sdxl can do without potentially introducing distortions in the images by setting the width and height to something smaller okay all the settings are good to go let's click generate knowing that generating the sdxl images is going to take some time I am going to skip ahead and show you the end results but just for reference all images generated with the version 1.5 models only took a total of 15 minutes but the sdxl images took a total of one and a half hours here is the completed XY plot with the models laid out horizontally and the Styles going vertically note all these images were generated in one shot meaning I did not cherry pick the results so we will see some images with artifacts but I think this is a fair test for all four models let's take a look at the images from the top as we go along I will keep score one point for each image that stuck to the prompt fairly well and generated an image with little to no artifacts for the first set of images I prompted the models to generate a girl smiling and with a off-shoulder dress we can see that all models did well except for realistic Vision here it generated two women in the image this is understandable because I set the width and height to 1024 by 1024 but for version 1.5 based models since they were trained with a native 512 by 512 resolution a larger size can introduce this type of double artifact for the second set of images I prompted Six Women smiling posing for a group photo and only the sdxl model came close to this prompt this is one of the weaknesses for the version 1.5 models which is when you prompt for multiple characters in the same image it has trouble giving you the right number but it seems like sdxl is better at this that's a point for sdxl for the third set I prompted for one girl smiling full body shot outdoor lighting and depth of field all the version 1.5 models did well but sdxl did not give me a full body shot this could be due to the seed value maybe a different seed would give a better result points will go to the version 1.5 models in the next set I prompted for a Golden Retriever with a chew toy in its mouth in a field and look at the mess that we got again I think the artifacts were mainly due to the images sizes being too large for the version 1.5 models the only Point here is going to ease sdxl for generating a normal image for the the next set of images of a restaurant the main thing I wanted to test is to see if sdxl can put legible words into the image and look at that sdxl was the only one that put an open sign in the window of the restaurant definitely a win for sdxl for the sixth image I prompted for snow-covered mountains two moons in the sky a floating Castle on top of a lake and I would say dream shaper and sdxl are the clear winners here as both images had all the elements in the prompt and the images have great compositions technically we are missing a moon in the images but I will let that slide next I prompted for a time-lapse photo of a skyscraper downtown night time and airplane in the sky looks like dream shaper and Mina mix made great time lapse images but forgot the airplane realistic Vision gave me two airplanes sdxl delivered a great image with all the objects but didn't show the time lapse aspect no points for any of the images on this one up next I prompted for a tilt shift photo drone shot of Paris what I was looking for was some of the landmarks of Paris the first two images delivered the right style of buildings but seemed to be melting the landmarks together only sdxl gave me what I expected to see albeit the image did have some trees in the river but still good job sdxl for the next set of images I wanted to see an interior shot of a modern bedroom in a brochure style photo we got some nice results from all four models but I have to give points only to maina mix and sdxl for the high levels of details in the images seems like sdxl is delivering its promise of generating highly detailed and complex images with very little prompt guidance next I prompted for a product image of a sports car as we can see there are unfortunately artifacts in the first two images realistic vision and sdxl both get points here but only half a point for sdxl since the car is lacking some detail and polish in the second to last set I prompted for a product photo of a gourmet cheeseburger I think I like the dream shaper image the best all the other ones have some flaws in them lastly I prompted for a documentary style macro photo of an ant carrying a leaf unfortunately no points for any of the images here but I think sdxl came the closest to having a good image to summarize even though my little test here is a bit subjective we can still see that sdxl has delivered on the Promises of generating highly detailed and aesthetically pleasing images with very limited prompt guidance and we saw it has some clear advantages over the version 1.5 models in a few difficult aspects of image generation the stability AI team also promises that sdxl is going to be easier to fine tune which is great news because seeing how good the base model is already I can only imagine what the Wizards of the AI Community can do to make the fine-tuned models even better before we end this video here's a bit of bonus content let me show you how to use the sdxl refiner model we are going to load it in the image to image tab drag and drop your sdxl base model image to PNG info then click on send to image to image this will ensure you carry over all your prompts and image settings remember to switch your checkpoint to the refiner model don't leave it on the base model then set resize mode to just resize since this is a portrait check restore faces set sampling steps to a higher number let's choose 80 and set denoising strength to somewhere between 0.20 to 0.30 here I'm going to set it to 0.25 also be sure to use a random seed then click generate skipping ahead to see the final image on the left is the original and on the right is the final image after refiner model we can see that the previous artifacts in the image are gone and we are left with a very nice clean and detailed image and that's how you can use the sdxl refiner model to enhance your base model images I will leave you with some of the other sdxl images that I have generated and the final images after running it through the refiner model [Music] [Music] foreign [Music] I hope you enjoyed this video and found it helpful please like the video and subscribe to this channel and I will see you in the next video

Channel: Keyboard Alchemist

Views: 9,418

Keywords: SDXL, SDXL 1.0, Stable Diffusion, Stable Diffusion Tutorials, Automatic 1111, A1111, AI, AI Art, Tips and Tricks, Tutorials

Id: nLz9Hksq71I

Length: 16min 42sec (1002 seconds)

Published: Sun Jul 30 2023