ComfyUI : EASY Face Fixes & Swapping my wife's face into images!

hey everybody Scott Detweiler here so today we're going to take old images from 1.5 and we're going to convert them to sdxl and fix the faces so the goal is to fix the faces because there's a lot of wonderful images that 1.5 has created at least with the base model but I didn't like the faces so we're going to do face refinement in here and then we're also at the end we're going to throw in Roop which is the ability to take an image and replace the face with a face of someone else so that's kind of a fun little thing I've been doing a bunch of images of my wife and throwing them at her and she thought it's pretty cool so we're just going to play with that a bit but beginning of this video we're going to walk through how to create a template so the templates are something we keep doing the beginning of this video has always been the same thing it's always create the base graph and then go forward and I'm dropping different tips and tricks as we go but we want to do is create a template so we only have to do this once from this point forward otherwise these videos are going to get really long so we're going to build the ultimate sdxl base here it's not going to take very long and then we'll be able to use it over and over from this point forward so again if you're a sponsor of this Channel or greater I'm going to put that in the um the post area and YouTube so you can go ahead and grab that graph and then import it into your comfy so thank you very much you guys for supporting the channel you mean a lot to me and again we couldn't do this without you because of Revenue on YouTube is not fantastic so I really appreciate what you guys are doing for me so let me know what I can do for you so we're going to go ahead and jump right into the graph here we're going to build this out again the idea is to handle a 1.5 SD image and then convert it to an SD Excel with a better face and then we're going to go through a couple because there's some really janky faces and I need to fix them so this would be a really entertaining process I'm sure I'm not going to cherry pick the results either so we'll see just how it goes all right as I said we're going to try and build out a base template we're going to use over and over again again I'll throw in some tips and tricks as we go so it isn't boring if you watch me do this a thousand times so first of all load node loaders and then load in our checkpoint now we're going to want one for our base and we're going to want one for our refiner okay now we've got both of those loaded we're going to go ahead and process the clip for this number oh before I get going too far here inside of your manager you are going to want to load in a custom node and that is the impact pack it's right here it's loadment number two here Dr Lieutenant data the same person who brought you the wonderful manager we're using here also has this impact pack we are going to be using the hell out of this thing so make sure that you load it we're going to use it a ton so again impact pack and it will install us a couple different models and so on but just let it do its thing because we're going to again use it a lot so the first thing I'm going to do here is get our clip model loaded so sdxl and we're going to load the one for up here and then we're going to load the one for the refiner as well there are two different ones now the difference between these is that the one for the refiner has an aesthetic score so the aesthetic score here for the positive is six and for the negative is 2.5 now again these are numbers that internally at stability these are what we use but again that's probably changed over time and you do you you know if you feel you have a better idea feel free I mean there's no stopping you from doing whatever you want to do here it's kind of fun to kind of experiment and see where you can go with this so I'm going to go ahead and color these up just so I remember them this is going to be the red this is our negative and the greens are positive and because this is going to be a template I'm going to use in this point forward I might as well go ahead and retitle these as well that way we have this forever all right so here I have my positive and my negative for the actual base now the difference between the clip G and the clip L that clip G is kind of the one that understands you better it's uh kind of think it might be more rule following it kind of gets you it speaks better English uh than the L but the L is more of your artist or your artistic your feeling of that type of thing so use the combination of them together now my negative prompt I'm not going to use the L at all I'm just going to stick with the G and the refiner only speaks G it doesn't speak any l so we'll just be working with though here and again we're going to color them and title them and go ahead and hook those up and that's basically it we're going to use this over and over again right so to make this really nice we can actually just highlight all of these and then you can right click anywhere and go down to a line and we can align them to the right so now they all line up and we don't have our our OCD triggering constantly there we go now we do want to say this is a template and to do that you just right click and select everything that you want to here and then we're going to right click but not on top of one of the nodes this is key otherwise it won't work off of the nodes once they're selected and you see that there's save selected as template so we're going to call this the ssdxl base so now in the future we can just go and right click anywhere on our graph node templates SD Excel base and boom there we go so we don't have to do this over and over again but again I did want to show you how to do this because it is very handy for no groupings you tend to use a lot all right so as I said what we want to do is we want to take 1.5 images and fix the face so the reason we did all this first and we haven't made a template with anything else is because this point is where the graph changes and we're going to use again a very cool thing inside of the impact pack called the pipe and a pipe basically takes a whole bunch of these noodles and combines them into one line that has done our pipe so that we're not dealing with like a bunch of buses or or other lines stringing everywhere and I absolutely love this concept it keeps the graph nice and clean but yet allows you to break things out when you need to so if you double click and type in the word pipe you see two detailer pipe sdxl is the one we're looking for and it takes all of these inputs to converts them into one line which is pretty sexy we also have the ability to add a Laura if you would like from here so let me go ahead and hook these up real fast okay now there are two others at the bottom here that you may not be familiar with that's B box detector and Sam model so what I want to do is load in the of the B box which is going to be our bounding box for trying to find the face in the scene so we're going ahead and grab that now to do that we're going to use another one of the things that includes in the impact pack and that's called ultralytics so you type in ul and you're going to see ultralytics detector provider remember things that start with the provider kind of come before the node but are used in a lot of situations the provider is used every single step but in this situation I think it's just used at the beginning so in this in this case we want the B box here so we'll go ahead and grab that and it comes with this one here called the face yellow M8 and that stands for you only look once by the way not for you only live once it's only you only look once we want that one and then we need a Sam model so we'll go ahead and if we drag that out by the way you'll see that there's a Sam loader which pops right up so that's super easy to figure out and we're going to use is the one that comes with it again here and that's basically it we don't have to come back to this other than to change the prompt so this will make life a lot easier for us because we only have one line to go from here on out okay now that we have this in place we can bring in whatever image we need to from 1.5 or whatever model could be my journey could be a photo whatever where you want to replace the face so we need to bring this detailed pipe out to do face detection so we bring that out we see here here's a face detailer pipe and look at that it is a k sampler combined with some other little toys to help us use this bounding box and the segment model to find the face and then replace it so this can add details to a face that's it's full job and it's going to be used down here in this model here this you only look once base model which is a fantastic model by the way for fixing faces so this key right here is all we need now this should fix most faces in one go but we could do multiple steps with this we can combine this with another one and kind of pass the ball or the face as it were from one to the next so from here let's go ahead and load in a model I'm sorry load in an image so image and then we want to load an image so this is the one I want to bring in here I love this image it's fantastic the only thing I like is uh whatever Expressions going on in her face there um great picture like I really like the whole fold in the dress across the steps the flowers this is the 1.5 base model by the way I really use that base model a lot you enjoy it and it doesn't have some of the biases or uh forgetfulness of some of the higher trained models so this is one of the reasons I really love it so we're going to start from this and this is the image we're going to bring in now we're going to put it over here which is a little awkward and we're going to do that because I want to kind of have the beginning middle and end pieces over here so that's why it's positioned in this awkward Manner and then from here this is the image that's going to come out of this first face detector here our face detail I should say so we'll put this over here and we'll preview it because we may not choose to save it we'll see how it does but would be interesting though is if we bring in like what it looks like if we just crop out the face what's that look like well let's show that uh so from here the number of steps we want well let's go ahead and crank this up to say 35 uh Euler is fine and denoise is the critical part here man this is the face so it's going to look at this shape and the direction of light and so on a 0.5 is probably fine you could probably drop this down to 0.4 and still be pretty good if you make it too high it's not going to have coherence with the rest of the image and it's going to look like a face that we just pasted right on top of a random picture so not a very good so we want to keep it like this so This should look pretty decent out of the back now there's a lot of other settings in this thing there's a whole lot of things going on here the grid size Max size there's a lot of documentation for these to determine how best they work together I just leave them at the default and if you're not getting the results you want then I would dig into it and try and figure out if there's things you can change if you're dealing with something that's out of scale this force in paint can help so that's the only one I would recommend that you may toggle on and off but for the most part most of these settings work pretty well and then obviously refiner ratio is a big one as well so how much of this scxl refiner you want to have a play in here and 20 is probably fine in fact probably recommended so 0.2 or 20 would be where I would go so that means if the refiner is going to have 20 of play in the image so if we go ahead and warm this up by the way let's go look at our prompt they just have elegant woman in the spot there and then beautiful woman I have no negative here we can throw one in like the word blurry which I really don't understand why that's so popular but we'll just leave it there and then the positive here for the refiner elegant woman's fine and then no negative I really don't use negatives as much with SD Excel I don't see the reason and then here if you want to Laura go ahead and throw that down I'm not going to use one and so let's just see what this looks like okay so we see a close-up of the face here that's much better than what we had before so a big Improvement there and well let's improve our view here by scaling this up I wish there was a really nice button to kind of make this so that we could just kind of maximize the size of the image uh but there's not so we'll just do this fiddling around that we always do until we get up where we want so I want to have kind of an A B comparison by the way they did set this to the middle Mouse button now moves the canvas or the nodes around which is really nice before it was hard to get a space where you could use your mouse so this is okay let's try another prompt here or another run okay so here's where we landed it looks pretty good that's just the face obviously a good improvement over whatever that is so this looks pretty good happy with that so came the improve upon it well we could do another one you could just fit another B sampler or base detailer in the middle pretty simply because it's using the pipe so we can continue on by just throwing the pipe over to this throwing an image that came out of whatever the first one was that'd be this one and we would have another image here so this method and maybe we say this one because it's going to have two shots at making the face correct the first one is to fix the majority of the errors then kind of hone in on it and then from here maybe 0.4 maybe just go with a 0.3 a little bit less because we're getting closer to what we want and run this prompt through I did not make a face um detail we should maybe pull that off but we kind of see here that it did indeed add a little bit more volume to the face and so on it's still not perfect but again we do have a random seed here so every time we select that and run it it's going to produce a different face here which we're going to build upon later so this step may be unnecessary it really depends on how messed up the face is when you're arriving so a lot of the 1.5 models right out of the bat were just not great at faces so sometimes this double hit on them is working pretty well let's just go with one here and now let's uh let's do the root thing so Roop is pretty cool it allows us to add a face uh that we know into the image and we're going to use the image that comes out of this to put into that roof now that requires another custom node so let's go into our custom nodes here to our installed we can see down here we have this one the reactor node 0.1.0 this is a group like Face Swap and this is pretty great so if we check out the GitHub for it it has instructions for you to follow if you're using an automatic 1111 you can do that as well and requires installing a lot of other tools which I did do because I was trying to get this to work with automatic 11 element to begin with eventually though you just come down here to the simple complete directions and it's really simple throw this in here run install.bat and it does all the good news for you now because I did this already I really don't want to show this and say hey this works automatically if you have issues I would post them up on GitHub with the developer so we can work with you on getting through those because I'm really not helpful there because I try to do this the hard way I guess before I get to the right way once you have that you can go here and find the reactor face swap and this is pretty interesting so we're just going to pull this image that we produced here into the input and for the source we're going to use a picture from our hard drive I'm going to use one of my wife actually okay I have this picture here a great thing about being a professional photographer is I have a lot of great pictures of my wife so I'll put this up here and we're going to do is we're going to swap out the face the zero face in the input so it's the upper left hand corner is zero for each face you so as you go from from left to right it'd be zero one two obviously and then from top to bottom you would also increment so it's possible for you to take a face that's specific in either the source or the input and swap them in this case it's the only face so it's going to be the zero position in this image and the zero position in this image which should be enough and you can also gender detect as well we really don't need to do that here I'm going to get rid of well I guess we'll leave all these here why not but this fixed face I think is worth applying to the roof first I think if you do the Rope against the you know torqued up face that comes from the 1.5 or wherever you're getting it may not be a great addition but with the face is more normal it just has a better chance of success so let's pull that out and let's save that one because I'm sure she's going to want these so let's try that now okay one thing to be aware of too when you're doing this is you want to find faces that have a similar expression to the Target so if she's grinning your butt off it may not be very good in this case it does look like her it's not a great match though so let's just spin the wheel again here I'll make sure I don't have this set on fixed I don't um I want to make sure I have enough noise so I'm just troubleshooting this right um a 40 noise is pretty good against that so it's going to give us this face which looks pretty decent we're going to take that result this result pushed up into the face swap bring it down now this face against this face will result in a slightly different image every time so I would again work with it until you get something that works and looks more realistic compared to the person you're trying to inject into the scene again expression direction of lighting hasn't mattered as much as I thought it would it tends to fix that as it's applying the face change so that's pretty decent I mean it looks very much like her so that's decent but let's try one more um let's actually go for a different image completely here and see what we have we can do another demo with let's just run with this one so when I just dragged it in here and plopped it right into this load image which is the other thing I really like about the solution is we can just kind of funk it in here and it will work um ideally we would convert this to an sdxl ideal resolution before we continue to work this through this model for this case I'm just going to demonstrate how this works in general but all the little tiny nuances we've worked with in previous videos obviously observe those if you're trying to make a permanent solution let's just see what this looks like and it obviously did a nice improvement with the face here so this is the original this is our improved face and then there's one with my wife's face on it it actually worked out pretty darn well uh so yeah that that works great so Roop is kind of a fun little thing to throw in here uh this in this case the face wasn't pretty messed up to start with so if the face is really torqued I would really impress upon you to use this uh face detailer and especially if you're not applying a rope and you're just trying to fix a face in general like if you're one of those people is a big fan of having vacations of vacation photos of people you don't know and that's your goal behind uh using AI great I'm more from the painterly aspect I really like these images like painted and more ethereal so that's the direction I'm going with these so it's pretty cool to be able to apply a face of someone I know to one of these I had this really great idea of decorating the house for Halloween and replacing all the photos so we're gonna have a Halloween party and replacing all the photos with like creepy photos but with all the friends that have been invited so when they're walking around looking at the photos on the wall they would all be like oh here's a bunch of people around a casket that I generated in stable diffusion but I replaced the the pictures with their faces I thought would be really hilarious so we'll see however that goes over anyway it's a good example of how to use this a face detailer as well as the pipe we're going to use this pipe a ton and then again how to use the template so if you want to bring this up again now this is something you're going to use a lot you can certainly create a template of this entire thing but in my case I really don't need it that way I tend to just do this over and over again and then from this point forward it's a random depending on the type of solution I'm trying to solve or the type of problem I'm trying to solve so there you go hey everybody thanks for hanging around today I hope this was entertaining and you learned something from it let me know in the comments what you think and again everybody who's a supporter of this channel is sponsor level are greater again click the shiny join button down below and then I will make sure to get you that graph from today so you can go ahead and play with it locally yourself so let me know what you think in the comments Everybody Take Care stay safe and I'll catch y'all next time

Channel: Scott Detweiler

Views: 23,603

Keywords: stable diffusion, stable diffusion xl, ComfyUI, stable diffusion tutorial, AI art, SDXL, comfy tutorial, sdxl tutorial, sdxl template, comfy roop, comfyui face fixing, face fixing, roop

Id: ekofgf9T-9c

Length: 19min 27sec (1167 seconds)

Published: Fri Sep 08 2023