How to use ControlNet. ControlNet for Stable Diffusion Tutorial.

Hello friends do you remember control net do you remember having this and turning it into this and this and this and this and this again power of control net now updated to control net 1.1 with a lot of quality of live updates let me show you how to install it what's new and how you can use it to perfect your images let's get started and in honor of my biggest donor gland what's the best thing about Switzerland I don't know but the flag is a big plus alright so first let me show you how to get this installed and working properly if you have an old version of control net you can just go inside your extensions here and you can check here's my control net make sure that is Click check for updates and you can see here it says web UI control net behind and I will just press apply and restart UI and that will update my control net to the latest version now we can see down here I have control net and it's very version 1.1 now if you don't have it installed you need to go to this web page I'm going to link this in the description below and I have an install guide here it's very easy you don't have to follow everything here you just have to take this copy paste this URL go back into your stable diffusion go into the extensions tab install from URL copy that in there press install when you've done that it will be in the list here now if you don't see control net here you need to just restart your stable Fusion altogether by closing down your command window and open it back up now if you have no idea what I'm talking about then you have no idea what stable diffusion is you're going to need to install that and you can follow my ultimate guide for that if you're on a computer I'm going to link it up in the right corner here right now if not find it in the video description the next step is you're gonna download the models and if you followed either the link in the video description or in the link that you can see from the control net page on GitHub you're gonna get to this page and what you want is the python files here the bth and you can download all of them if you want to now they're fairly big no not big for stable Fusion models but they're 1.45 gigabytes each so if you don't have this the space to have all of these I'd recommend getting just a few and then I would get the depth here I would get the canny this one the open pose and the scribble now it's a matter of preference but those are the four I would recommend that you start with if you don't have the space you don't need to download the yaml files because they will automatically download for you when you've downloaded those files you're gonna go into your stable Fusion web UI folder then you're gonna find extensions you're going to find reviewer control net models and then you're just gonna drop them all in here and as you can see I have the yaml files here that I have not downloaded them manually stable Fusion has done that automatically for me once it has detected that I have the pth file and loaded up control net restarted and everything once that's finished you will have control installed and it will look something like this you will have the little tab here you can press that and you will have most likely control net unit 0. if you want multi-controllet meaning more than one controlled unit here you will have to go into settings up here find control net down to the left and then it says there multi control net Max models amount requires restart and I have set this to 2 now you can just slide this to however many you want just remember to press the apply settings and then restart once you have it set up like this this is where you will drop your input image and for the this I will show you how to get that you can either get any image you want like a photograph but if you're looking for something more specific I would recommend that you look at post my art which is also be in the video description below here you can load up any model that you want you also have some pre-made scenes that you can load of of poses and here's some kind of fighting scene of people jumping and then having a sword fight here's someone dancing but what I liked most about this is actually the animation so let me show you in a bit let's let's say I have this character here and I want to put this into a post let's do the robot chair for example [Music] anyway let's move this around let's say oh let's do a kick here when trying to move up the leg and it's oh it's all wonky whatever it's moving around but oh this is hard well well it's not super hard but it's a little finicky if you haven't used these kinds of programs before now we have some kind of a kick here would you say that this is a good kick I wouldn't say that this is a good kick yeah you can see the foot here and the leg here it's all messed up so I could spend like 5-10 minutes to fix this or I could go into the animation so if I select this or just press the little character down here if you go to the bottom left you have animations and poses so I mean first off you can just pick a pose so let's say I take this one for example I mean that's great I've now written wave let's take this one here and now we get a full animation and you might think well what am I gonna do with an animation Seb well here's the kicker you can just drag here and find the perfect frame that you want for your animation or well your still image so let's say we want this here so now we have a perfect pose that feels a line because it came out from an animation let's try another one so here we have another animation and it's just a simple greeting but let's take this one there we go and now we have a pose that we might like what we can do is we can press the export button up here and what's great about this is that you can actually export to open post now you can export the image directly like this but then you would retain a lot of this in the depth map or the Canon with the open pose you're gonna get just the pose you can scroll in and out like this on the side but if you scroll here on little mouse nothing's happening so just remember that in the same way you want to move the output window you need to select the move here if you don't you're just gonna rotate so let's say we want this we press export open pose And Hive save this as wave now if you go inside your stable fusion and we drag the wave image in to here you have this exact pose now we need to remember to click enable here on the control net and for the model here we're gonna choose the control V 1.1 SD 1.5 or 15 and then open pose now there's a lot of numbers and stuff here so first is control Nets then it's the version which is 1.1 then there is a letter which is either P U or EPS production that's basically a final one experimental I think EU was unfinished or untested doesn't matter p is the most final one and then we have the stable Fusion model it works for in this case the 1.5 as you can see down here I have a model that works for stable Fusion 2.1 for example and then the last here that's the kind of model that we're working which in this case case would be the open Pulse now since we already have the finished open pose here we don't need a preprocessor if you would have a photograph you would need to use the preprocessor here for example open pose full or open post now wait here that's basically waiting the control net model so the higher this is the more it will follow this open post so if you set this to lower it will kind of follow it but not fully one is usually a good value here the starting control tip and the ending control step that's where the control net will start so now we have it starting at Sierra which is the beginning and it ending at one which is well 100 so it's gonna run for the full duration if I set this to for example 0.5 then it will not run control net until 50 percent of the render then for the remaining fifty percent it will run control net likewise you can set this to zero and this is 0.5 then we'll it will run control on it for 50 no control at in the end we want it for the full duration now guest mode has been renamed in the control node usually have it set it balanced but you can also set my prompt is more important or control it is more important most of the time you're not going to bother with the resize try to work with the same resolution here as you are up here so let's try to create something let's say woman waving we're gonna up the weight here a little bit and then we're gonna add fault negative give the digitaloid oil painting and Skinner answer and all of these styles are the preset styles that I use you can find them for free in the video description and after sampling steps to 25 running Euler a we're not running resource phases or high-res fix however high-risk fix is supported in control not 1.1 and as you can see on our results here we have two women waving and they're very close to the open pose that we had in below here and control it now the handsome face are kind of messed up but um that's not really what we're looking for in the 5 12 by 5 12 here we're trying to find the pose and if we're gonna continue with the image we would probably upscale it through an image image pass to double the resolution and then start in painting the details so that this is mainly to show you how open pose works now let's say that we want to improve on our character here so I've sent this into image to image and I've also loaded control net here so these are similar to the settings we have before we have the open pose we have the open pose here and I've enabled this now I'm just gonna up this to 1024 by 1024 we're going to lower this to about 0.6 I'm changing this to a slower sampler I want the 2m cares for this and wanna up the steps just a little bit now as this is a slower sampler I use really don't use it for my first generation step in 512 by 512 but for finishing touches when I found the composition I use this to try to get as good an image as possible so now we are running this in 1024 by 1024 you can quickly see if you compare just in the generation here this is much better even the hand is better compared to the left one and the face is holding together great now you can see it in full resolution yet as it's rendering but we'll get there in a second and even the second image is turning out much better it seems so we have the first one here face is kind of good the hand is okay-ish it looks like we have one two three four things there I think that one is just actually fading out into Oblivion this one's much better when it comes to the hand just some artifacts down here so this really is the power of control net and with the update to one 1.1 it will be much easier to use will not easier but we get some better results you might be asking well I've seen all this what are you talking about said what's what's even new in 1.1 well let me quickly show you on the page here we have new features in control net extension 1.1 and it has perfect support for all control at 1.0 1.1 and T2i adapter models so you can even run the style adapters and and other models in 1.1 it has support for the high-res fix something we mentioned earlier now I personally don't use high-risk fix a lot I know a lot of you like to use it and that's fine all good it's just a different kind of workflow what I do is what you just saw text to image in the lowest and then image image to a higher s high-res fix is basically doing that in one step now it says here perfect support for or well basically mask so something I enjoy which is in painting now has control that support so that's a great feature they have something called Pixel Perfect mode and that's the little button if you go back down here it says Pixel Perfect and that's if you have for example something that's different in size to what you're trying to do you can fix the Pixel Perfect thingamajig here and yeah they write here now if you turn on Pixel Perfect mode you do not need to set pre-pressor resolutions manually the controllers will automatically compute the best annotators or solution for you so that each pixel perfectly matches stable diffusion and that's mainly for this let's say we have the can here for example the resolution here now our image is 512 so it doesn't matter but if it's not and it says user friendly graphical user interface and pre-processor preview we reorganized some previously confusing UI like canvas with height for your canvas Etc and I agree this was confusing I made error in my first controlling video and I'm sorry if you've been following that particular advice and they changed a couple of bugs in the guest mode etc etc and by the way here's a quick example of the guest mode so here's an input if you have control net mode balanced it's kind of listening a little bit to the prompt a little bit to control it view control mode my prompt is more important you see this is stylized prompt and in control mode control net is more important it listens more too well control it and control the input now let's say you have a photograph or well it doesn't matter anything you can use that for your control lit as well let's download this image here we're gonna reset everything how pack enter text to image we're going to drag our control net image here we're gonna enable we're gonna set this to Canyon I'm gonna activate Pixel Perfect here I'm gonna load the Kanye model there and we're gonna have let's have beautiful Studio Ghibli house in Autumn gonna have to wait it by control Arrow up we're gonna load the same as before we're gonna load the gibbly digital painting and the default negative let's run four Images here and we are generating so now we have our four results we have the preview of the the canning model so this is the what control it uses let's look at what we had here so these are our results let's take take this one here and send that to image to image again we're gonna up the size change it to Keras I'm loading my styles again now if you have the Styles up here you don't need to reload them and for the image image step you don't have to use Ctrl net again so this time we're gonna skip that and here we have our finished image of the house and if you compare that to the original input here I think it's pretty good pretty cool now you can get more or less close to the inputs depending on the settings you have below anyway I hope you had fun and I want to thank the sponsor of this video which is post my art for um first of all helping me with the channel and my videos and also for supplying this great resource for users having this together with control net really is very powerful I hope you had fun in our little video here uh as always have a good one and yeah

Channel: Sebastian Kamph

Views: 259,931

Keywords:

Id: WZg3e6B2yPQ

Length: 16min 23sec (983 seconds)

Published: Wed Apr 26 2023