ComfyUI - FreeU: You NEED This! Upgrade any model, no additional time, training, or cost!

hey everybody Scott Davila here quick video today showing something I thought was really cool so this new node just released today is called free you and it allows for you manipulation and they're basically saying it's a free lunch meaning it's pretty much not going to cost you anything to implement so the concept here is we're going to take the unet which is the core of how stable diffusion functions and then these skip connections that cut across to basically add detail back in as where on the decoding side but what they're doing there is they're saying let's re-weight this so the backbones and the skip connections themselves those features that come across more the high detail stuff could be re-weighted so the contributions that those will make can be changed and they say this is free it cost us nothing and yet the result is substantial so let's see how to hook it up I'm going to start here with my basic xdsl graph now we talked about building this a few episodes ago and it's pretty simple in fact we're even going to get rid of the refiner for this one so it is basically our positive and our negative here and then I put a note over over here in the graph that I have that's down in the post area for all the people who are members of this channel which by the way thank you so much for your support as I said we couldn't do it without you this graph is available for you there and I need to put some notes here so you note what the clip G and clip l are in the xdsl models in case you forget so remember the refiner only uses clip G so I kind of put that in there for you so this is a very standard graph we have our we're loading our checkpoint here which is our xdsl base we're loading the resolution which I'm using one of the comfy math uh nodes here that basically allows you to pick a resolution that's safe for sdxl and then we're using that to drive both the sizes of the the targeting resolution of our our condition here as well as the empty latent and then I'm obviously putting in here the size of the batch I had done this because I was doing something else which we're going to cover in another episode and then we're feeding that into a k sampler and of course out of this we're going to bring in our vaed code and our vae all right so this is pretty standard and we'll let's preview this one here what I'll do is I will put in a quick prompt here I'm using a photo of a western movie clip a show down the middle of a Dusty town and the clip L here we'll put a photo of a dirty western town where two people are about to have a pistol door all right so we've got that done and I didn't put anything in the negative because I don't so what does this new thing do and we're just like oh it's actually really easy to hook up but I want to show you this because we're going to actually use a comparison here so you can see what it's doing so the first thing I'm going to do is I'm going to double click and type in the word free and you'll see that free U pops up this is a core component in comfy however you will need to update your company so make sure you're up to date otherwise it's not going to show up and you're going to wonder why that'd be the reason so I'm going to do here is I'm going to use the model and just feed that into this and we're going to take this same K sampler and we're going to feed this in now what I want to do for comparison is I want to make sure that our seeds are the same one so I'm going to actually do is change the seed here convert it to an input in both of these and then just drag it out and we'll put a node here add a node and we'll go under utilities we'll add a primitive that primitive I will be whatever whatever we attach it to in this case it's an integer and we're going to feed in our random or we're going to feed this to both of them so we get the same seed that way we're comparing apples and apples basically and then obviously we need our prompts we're going to use the exact same positive and negative here that we did there and we need the same latent so no variables have really changed between these two things other than this thing being in the way so that's really the goal here is to see what is that it's doing obviously we've got a bunch of noodles running around here and it doesn't be neat because obviously I'm just demonstrating this but I didn't want to show you what it looks like so we'll just do this okay so let's take a look at what happened here now we did not adjust the settings yet we're just doing one just out of the box to show you what it's doing so what's allowing it to do is to carry forward some of the detail that might be lost now we're using the wrong settings here but I kind of want to show you the two images are going to be different right the Hope here is that we get more of the detail here out of it and that's that's called the higher frequency parts of this and then we obviously want to keep the lower frequency like the building shapes and so on you see in this one for example it turned it into a mountain versus this one it's a giant edifice there so let's go adjust this for the model now there are different settings for each of the different models from 1.5 to sdxl and those are on the website there and I'll put the link to the get over there so you can grab it now goofing around here locally and this is obviously not scientific at all and I have a lot more to play with but so far I kind of like these settings this is not a 1.1 I think so I'm playing with this trying to figure out what settings work the best and I I really don't know yet I want to create a big graph and I'm going to actually have a whole video on graphing or creating XY plots so you can find what settings make you happy but that's another day so that's going to be quite a video all right so let's go ahead and adjust that now let's do this again all right I don't like that when I'm gonna go back up here and adjust this from one to a 1.0 and do it again because I didn't like the way that one turned out as I say this is still kind of a little bit of experimentation all right let's take a look here okay we have a lot more detail especially in the ground and the outfits which I really like and the faces look a lot better as well it still has a bit of an illustrated feel to it but I think we're on to something here this uh again this cost us nothing but I think we have a better model the hands are better for example I just think overall this is a really interesting new way to look at models and I'll see how much we have to apply this in our other settings so far this morning I played with this quite a bit and I have to say that 99 of the time or 95 of the time I prefer the model that comes out of this than I do the one without it so I think it's a good step forward again I'll have to play with it a bit but you go ahead and try it out so I'll put the graph of this in the community area for everyone who's helping out the channel again thank you so much for your support so that will be available for you to just kind of grab it and download it and play with it again it does come with company by default so there's nothing special you have to do but I'd love to hear what you think about it give it a try and let me know take care stay safe and I'll catch y'all next time

Channel: Scott Detweiler

Views: 27,872

Keywords: stable diffusion, stable diffusion xl, ComfyUI, stable diffusion tutorial, AI art, SDXL, comfy tutorial, sdxl tutorial, sdxl template, ai art tutorial, freeu node

Id: 8XJH6uZjNzA

Length: 6min 2sec (362 seconds)

Published: Sat Sep 23 2023