How to use SDXL ControlNet.

Hello friends what if you could turn this into this and this and this and this and this welcome back to an updated control net now working with automatic 11 and sdx L so that's some major news for your stable fusion and generative AI levers out there control net is probably the best feature of generate Ai and stable diffusion so let me show you how to get it running inside of automatic 1111 and sdxl but if you're on a confi UI or anything similar you can follow along as well and I'll just show you where to download the models oh by the way did you know that once upon a time there was a king who was only 20 centimeters long it was a terrible King but a great ruler hey so this is what we're gonna see when we're finished we're going to have an input image and then we're going to have an output image that is similar to the input depending on the settings we have first of all if you don't have automatic 1111 installed find my video guide on that in that corner top corner right now if you do we're gonna start with updating it's going to stable Fusion folder up here you're just gonna select everything and type git pull I will open a command window and will automatically update your stable Fusion now it will close instantly see if you want to see what's going on you can instead type CMD and then write it pull and I will show you already up to date so that was what was happening in the background we just went so quick and it kind of went away once that's finished and you have started your stable Fusion you're gonna go into extensions here and if you don't have control installed you're gonna go to this link here press the code up here and then this button so you can copy that and then you can go into your stable version you can install from URL just paste that in there and press install if you already have it installed you're going to be in this install Tab and just press check for updates and then you should see that your control net will need updating so this will be controlled here now mine says latest yours will say that it needs updating then just press here and it will update we're also going to download the models all the links will be in the description below and here are the available ones so there's a lot of files you don't need all of them so for example like the canny here you have just here you have the full the mid and the small then you have the Kanye again here the koi I made and here's a separate one by PSI which is made with loras and again you have like different versions of it so I'd recommend that you just pick the diffusers here so the candy and the depth I chose the mid ones if you have the space you can go with the full ones the mid are pretty good you can also scroll down and find the open pose I got the 256 lower one but you know if you have the space you can get the bigger ones they are a little better but the 256 loras are still okay the 128 are you know you're pushing it but if you really are lacking the space you can try them as well so I'd say those three are like the main models that you can try there's also a recaller and you know the sketch could be pretty cool to play with if you're just doing like basic sketches I wanted to get them into images so we're gonna take this and when you download them they're gonna go into your stable Fusion folder extensions control net models you just drop everything in here and once that's finished you're going to be good to go with your stable Fusion now I'm restarting it here to have a clean UI for you guys now control net is going to be down here and you have several controlled units and you can change how many of these you have from the settings tab so settings control net down here and you can see multi-controller control at unit number now I have it set at three I rarely do more than three most of the time just one or two and if I drop an image in let's say I have an image just on my desktop here so this is some sort of a woman robot and then you need to enable your control ads I usually set this to Pixel Perfect if the image is in different size compared to what you are or what you will be using or creating and then you can see there's a little explosion here now if it the candy for example if you press the explosion you will see the preprocessor working and this will be what control that is is working with so Kenny is working with lines most of the time you if you use the depth map you'll get like a 3D depth map of the image black and white well you see so depending on what kind of image or what you're trying to do there are different models and preprocessors to choose from and there is more info about that on the main control net page on GitHub and that would be this one there's two different one there's the the maker build this is one that you're installing and then there's the Alias aliasville whatever where you can find most of the info about control that's in here is we can see like some examples and how different models are being used here's the Kanye for example here's MSD lines there a lot of 1.5 control with models here as well so not just the sdx L1 so this is an example of old scribble for example which would be similar to the sketch one we have now here's an example of the open pose you can see that as well in R UI if you change this to open pose here you should get a little yeah so here's the post we have an arm the shoulders neck and a head and the face and everything but we're gonna start with depth for this one because when I keep most of the shape of this character and let's change this into woman ballerina and I'm just adding some default negatives I'm adding realistic photo portrait style as always the Styles can be found in the description using Keras about 30 steps and I'm adding the refiner here for sdxl we are running draggernaut XL version 2 and we are changing to 10 24 by 1024 and we're just running one image for now we are running the sdxl Vie and the joggers in Laura this is not required but it's just something I like to do to give a little better results and now we are running this generation we are starting to see what is becoming our image and now it will be very close to our original because we have quite High the control weight is one and it's starting from zero to to ending at one so it's running for the full duration now as you can see here we got the post that we wanted however the image isn't very good at all the dress is okayish the arm is a little messed up and that's because as you can see here it you know goes from the robot Hunters here and we don't want that on the regular arm so what we could do then is if you take this control weight and you lower this you could also change like the open pose for example but now we want you know more specific posts so just lower the control weights and maybe end the control Step at about 0.39 here so now it will start off not as strong and then it will kind of not as strong as previously but still follow the image and then give it more room to create our image or our woman here and we can quickly see that we are getting a little bit smoother arms here we might need to lower the weight a little bit more but uh we'll see I think here the control that kind of stopped and now it's filling out you know kind of the missing parts of the image we got the post that we wanted the stable Fusion helped us to uh well generate the rest of it that was lacking from controller I think in my mind this will turn out to be a pretty good image and as we can see on the result here we have similar posts or the same pose but our image is much much better now the face is a little messed up we can fix that in a second we have some grain from The Prompt so this is more of a like a film style image but now we have an image that looks very good and still Falls control it now this really is a controller tutorial but you know I could just send this into impact maintain the face here just to add some detail in the face so Woman face and we're changing it to only masked and the resolution is still 10 24 by 10 24. we're using mass content original lowering denoising about 0. let's do 55 so this should give us an image with a better face but still retaining the rest of the image that we got from controller now this step isn't using controller Tower is possible so you can see down here to actually load control net even in in painting and if we look at the finished one here we can see that we got a much better face compared to what we had previously now if you would use some of the other control net models like Kanye for example you saw that we had the kind of images here on the preprocessor so that will follow it will give us much more detail we can actually increase the way it air again we're gonna run this fully and it will follow this heavily if we change this into like night in full armor let's do a digital painting this time and we are gen rating and as you can see here with the results we didn't really get an item full armor and that's because the canyon lines are so strong that we almost got the same image again so remember that we're using canny that you know it's going to be super close to the original image you might need to lower the weight or or use a different model depending on what you want you might want candy for that exact reason that you're actually follow the original like a sketch or something let me know what you thought in the comments below what model are you using what do you think of sdx will control that as always have a good one see ya

Channel: Sebastian Kamph

Views: 72,286

Keywords:

Id: gkkfdWukb5I

Length: 11min 15sec (675 seconds)

Published: Tue Sep 05 2023