ControlNet Assisted Latent Upscale (Enhanced HiRes Fix) - ComfyUI Tutorial

hey what's up I'm poison Berry I make uh AI art with AI so here's a comfy tutorial on something I like to call a control net assistant latent upscale controlling assisted late and upscale sorry uh so if you ever use automatic 11 11 you are undoubtedly familiar with the high res fix option um if you don't know what that does it basically just takes the image that you've generated it does a secondary pass after an enlargement or an upscale of some sort and then it's basically just an enlarge and enhance um the problem with this is um it the late and upscale method at least usually requires a relatively high denoise around 0.6 um we'll just cue this up right now so we can see what's so this is what a latent upscale looks like hold on this didn't seem to go through uq key prompt cancel all right he prompts Okay so let's get our initial image I'm using the same seeds that I like I really like this picture so this is the initial image right here and then you late and upscale and now it looks like absolute garbage so this is the reason why you need a high D noise when you're using a latent upscale the problem with that is if you've ever used AI or image to image you know that denoise is effectively a how much do I want the image to be changed slightly so a high D noise equals an image that will be not drastically different but it'll be different enough to the point where you will absolutely notice and that's not always desirable because in this particular case I like this picture I really like this I just want an enhancement I want it bigger with more details but when you do the late and upscale it changes very drastically like the eyes are completely different the the we lost color on the hair around here I think we lost this Rose on her sleeve too the outfit's completely different it's not completely different but it's obviously very very different in the eyes too it just it looks I don't like the way that looks so that's why I recommend something called the control net assisted latent upscale and what that does is it it does what it sounds like it uses control net to assist the Latin upscale spelled that wrong sorry but uh yeah in this case I'm using just it's really simple just a standard line art preprocessor not the anime one um the anime one kind of messes things up sometimes um yeah it's just literally line art preprocessor apply control net into the model and then boom get the image so it preserved the colors we have I think a lot more color detail but it's it's still different it's still different but it's it's more similar like that rose that Rose we have this Rose we have the rose right there we got this one you can't see my mouse pointer sorry it added some makeup to it added some weird makeup to her face but yeah that's whatever I think I like it I think this is much better so I'm going to show you how to actually do this in case you don't want to study the uh study the actual node tree I'll just uh show you how to do it right now let's remove these groups I don't need to remove this one keep this one here let's remove that all right so we need to get these image previews in here again so we've got image preview image I like doing preview image instead of save image at the intermittiary steps so I could actually see what's going on with my workflow so we need a sampler so all this is just very basic stuff it's just you know your load checkpoint your positive your negative prompts your latent goes into the case sampler goes to beta code and you get an image right so then you take the late in from the K sampler there's a couple ways you could do this you could either take the latent from the case sampler and put it into an upscale latent or you can do um just like an image upscale and then plug the image node into here and then you still have to convert it back to latent though because it needs this the case sampler needs a latent image input so the way you do that is you go to but up latent VN code you plug in pixels plug in your V variational autoencoder and then the latent goes into there so there's two ways you can do it there's actually more than two ways you could also upscale using a model if you want but yeah just to keep it simple we're just going to the late and upscale yeah it's the thing that everyone does click late and upscale because that's what the Reddit guys say why not so plug that into here and so normally if you're doing a regularly enough scale you would just plug your prompt directly into here but that's not what we're doing that's not what we're doing we don't do that we are going to use pre-processors specifically the line art preprocessor you're going to notice there's one here that says anime line art preprocessor that is not the one we're going to use all right I'm going to show you right now why let's go to pre-processor again we get the the anime one we get the regular one the linear this is this the non-anime one is the one we use even if you do anime Arts why you're gonna we're gonna we're gonna show you you're gonna see right now so we got this clone this one put this down here so we can see the difference between the two got that pretty image boom that's what this is the anime line art preprocessor and dude there's so much noise these lines are so rough what is this what's this stuff on her face what are these like the spots man like what what is what are these dots what's this weird pattern what the heck dude this looks so much cleaner look at that it's so much better dude don't use this don't use the anime one I don't even know what this is for use the regular line art preprocessor okay so we have that we have the regular line art preprocessor not the anime even though this is anime art kind of we're not using anime all right I'm not saying that anymore so the next thing to do is conditioning and apply control net so the image it's not this image no it's this it's the preprocessor image that's the one you need for the apply control now so unless you're using the tiled preprocessor but we're not going to do that right now that's a different tutorial so image line art in here control net all right we need to go to loaders loaders load control net model and what's the one that we're using you have to match this to the preprocessor you're using or else you'll get problems you need line art all right so we plug that into here and conditioning that would be our positive problem so you just take that positive prompt right here put that bad boy right there plug this into here and then model comes over here so this gives us a latent output which means we need to decode the latent into a pixel space so that goes there get the V from over here put that into here and then get another you could do an image save if this is your last step but the latent upscale is never my last step so I'm not going to do that I'm gonna get an image preview make that bigger Q prompt fix let's raise the steps a little bit why not so I'm keeping the seed fixed because I really like this image I think it's really really pretty and if you're curious about the prompt I just made these words up I played Final Fantasy VII that's how I get Stardust it's uh red 13s I think it's one of red 13s limit breaks or something like that Stardust shine I have no idea I literally just made the stuff apocalypse rain literally just made it up nuclear explosion effect saw one once thought it was cool put it in the picture this is still generating if you think I'm talking to waste time nope it's talking to cover time because this takes a while to do all right so if it's it's just gonna give us a picture that looks like this I don't know why I have to wait this long 50 steps is probably too much for a tutorial then we don't need this down here either so ooh that is very very different that's because I use a denoise of one though okay so maybe denoise one's a little too high but still the composition is still pretty much the same we have you know the flower we got the other flower that's pretty much the main thing I was concerned about we're just keeping flowers but uh let's see if we can make this still look a little more like the other one change the sampler name and the scheduler see I don't know why but Karis care sampler with uh sde the stochastic differential equation seems to do a pretty decent job let's keep this at one actually it seems to do a pretty decent job at actually preserving everything that was in the original image I'm not entirely sure how samplers work in that regard but if my recommendation is that if you were doing something that involves image to image and you don't want it to change too drastically Keras use care scheduler and then one of these ddpm blah blah blah don't use Euler ancestral if you're doing in pain stuff because that it changes things a lot just in my experience if you have a different experience let me know in the comments maybe maybe I don't know everything who knows we'll find out 50 steps was too much apparently this is taking so long one iteration per second oh boy that looks worse bring this back down to 0.7 0.6 whatever I don't want 50 steps we're just going to do 20. 29. whoa all right there we go I mean it's just freckles now whatever there you have it that's uh that's my method for uh control net assistant Latin it upscale if uh if you learn something then uh cool if you uh want more tutorials then uh just

Channel: poisenbery

Views: 7,479

Keywords: stablediffusion, AI, img2img, text2img, comfyui, automatic1111, latent upscale, hires fix

Id: qbDm1s0ULpQ

Length: 10min 47sec (647 seconds)

Published: Tue Jul 25 2023