ComfyUI: Intro to Control Net- Sketch to Render (Part III)

in this video I will go over how you can develop more advanced workplace with compi and control net to allow more structural control based on input images and they are used in popular AI workflows such as the sketch to image generation when using the compi node based environment you can much more easily compare results from various control Nets and then choose the best ones control net is a core extension that is often used as part of AI generation workflows and so it is definitely worth getting acquainted with here I have a default workspace loaded up and for the checkpoint I have the realistic vision for stable diffusion 1.5 loaded in and for the rest I will take you through the basic setup first we will need custom nodes and models if you have compi manager installed as shown in the previous video it is an easy process we just need to activate the manager from down here and at the top install custom nodes we will be needing free noes so if we search for control net we will need the comp control net auxiliary pre-processor the compi com custom mode and the compi advanced control net I've already these three installed but you can just click the install button here and then you will need to close and restart the UI you then need some control net models to work with these if you want to install individual ones such as the scrible control net an easy way is a do Freedom manager as well so if you go to install models you can search for custom ones such as screw up you can restore them from here so you can see for the SD 1.5 version so this is good if you want just Lo the one by one however if you want to install all the control Nets there's a much quicker option and if you go to the GitHub control net site you can get a lot of documentation about the control net uses so they're very useful to understand the various outputs here is just using the example automatic 1111 web UI however you have the same functionality when used within the kuui node based environment if you go to the control net hiding base page with the link in the description below you can see a long list of ones that you can try out if you go to the top end and press clone repository and copy this you can just clone this into your control net folder in the comy UI you need to have G installed as shown in the previous video but otherwise you can go into your comy UI main folder P to models control net and then you type CMD and then paste in that copy line and hit enter these models are very large and there's a lot of them so take a while but when it's complete it will all be installed into your models control net folder and you can use them within the UI so you just need to restart and load it up again we now have everything needed to work so first let's start with a sketch to image workf we can delete this empty plated image as you will be starting of Zer let double click and search for load image and select this I would choose a sketch by the architect Paul Rudolph who created these amazing sketches for Mega structures and fist style buildings I will select my sketches of two towers and now just we scale it make sure that the largest Dimension is not larger than 10 24 pixels I also copy this and use this image or the case samp so to encode this we need to drive up from here and select ba AE encode and connect it up to the case something and then we connect up the VA e next for the prompts I'll copy in some standard prompts which I've taken from C Ai and then at the start with a positive prompt I'll just add a short description of the scene that I want so futuristic looking spey complex brutalist Powers now we can start with the control net loader I double click and search control net and select the load there and then from the drop down you can see this long list for the control net version 1.1 and we can start with the scrib bo next search for control net apply we can select that bounce version as a few more options and then we each connect up the positive and the negative problems to go through here which just HCK them up positive and negative and then the same add these into the cas samp positive and negative we can move these out so you can see what's happening better essentially everything's being piled through this control net whether than directly from here into the case sampler in case sampler itself I will just change the sampler name to DPM PP PRM all the ones here only quite good I've been using this one recently and for the schedu I use Caris give some realistic results and then is many need this D noiser I'll be using to influence how much the sketch will impact the generations so if we keep this to close so one and generate you see that there'll be more room for some creativity and won't follow the sketch as closely we can also add a pre-processor to see how the control net image is processed so if you double click and type in scribble you can see the scribble tree processor so whichever control net model you're using make sure it matches the pre-processor name and then we can connect the image over here and if we drag off on the output we do a preview image so this will allow us to preview what is happening in this control net which is very useful and then let's connect up the last noes so control n here and then image to the image you can now hit generate and immediately you can see the preview image of what the scub lines are doing and then you will see here that where we have the D noiser at one is got very creative image these Dynamic clouds blue sky and futuristic looking TOS if we down this D noise you see that will keep closer to the orual sketch so let's try five and press Q prompt now you can see this is much more sketch looking that's is a visual input so it's expect to have maybe 0.7 0.8 for the D noiser let's read this back up to 0.8 and then you see we have something in between what we had on the previous two generations so you can see the effect that we have with the scribble PR processor if you want to try out other control Nets and compare them you can copy and paste some of these notes so first let's just tidy up this a bit by moving out we should then select the nodes by pressing control and dragging contrl C then hitting control+ V down here and then if you hold down shift and move it'll move the group if you don't press shift you'll be selecting one of the notes and there we just need to connect up the missing noes so let's get the model lat image ba a yeah the image and then up cles we can change these but let's keep them the same for now a positive and a negative and let's change our control Nets to lot lot is gener quite good if you have a very detailed sketch which we do have a doing with and of course don't forget to change the pre-processor so let's type in line out and here's the line out pre-processor and connect us up you also start noticing that these SL start getting quite messy in the s quickly so one bit of advice is if you drag off from here you can always add a view out and it allows you to gives you more control but we'll leave them for now so we can try generate and see what we get you see a completely different effect by The Realistic light not control it and you can see this more artistic output it's looking quite nice what's also great if we drag this up we can quite easily compare our results and you can do this with all the other control Nets and you can see which one best suits the image that you want so I also do this quickly with the soft Edge and the death control Nets and we can line them up and compare them if we compare all of these previews together you can see the different effects the control net tab the control net here as you can see for the depth focuses small in the form so it's good if the focus are massing web details and even this soft Edge gives some interesting Styles so have a play around with them and after initial AI Generations if want to get more details it is always a good idea to use an AI up scale to of find these this can also be done with stigma diffusion which I'll show in the next video in this series this is just a simple example but you can see how you can also stack up these control nets for example if you search multi you can get m control net stack and you can plug in various ones and try different effects but this is great for creating even more advanced workplace hope this show control net was useful and I'll be deep diving more into some of these procedures so I hope to see you in the next video

Channel: Urban Decoders

Views: 4,746

Keywords:

Id: oGwTMmWXMjc

Length: 12min 6sec (726 seconds)

Published: Sun Feb 25 2024