ComfyUI Fundamentals - Face Restoration

hello and welcome to another comfy UI fundamentals tutorial we're going to be covering face Restoration in this session and to do that we're going to be including a whole bunch of well two or three custom nodes that are specifically designed for face restoration and there are ways of doing this um this stuff using just the standard kind of tool set and some basic kind of nodes but it's so complicated there's just no point really in me showing that um we're going to be using the same kind of setup we had last time you could load this with a load default node I am using sdxl base 1.0 but this will work with sdxl this will work with 1.5 this will work with any model you you want to use now I'm just using it because it's getting the image that I need to use to um demonstrate the the um process in order to do this process I've made it so that we've got two German soldiers from World War II facing the camera and standing so we've got their faces at a mid distance so it's that the distance at which faces tend to get messed up in SD is around this distance from the camera and um any closer and it's usually fine it's really messed up this this process of teaching will show you how to do this this whole process at further distances like than you would normally be doing but this will work just as well up close to the camera as well if you need it to so to start off you're going to need to load images you can choose some faces that are familiar to you that you just want to use you could choose an image with multiple faces in it but that's a probably not within the scope of this particular tutorial but this method that we are going to use at one point during this will be able to handle multiple faces it just uh yeah just just get two with faces and you can deal with that stuff if you want to go further than that all right and to start off it has generated this image and as you can see there's some Oddities in these faces it's the they're indistinct um there's some weird distortions in them um is it funny angles that kind of stuff uh you know not not Optimum you might fix this with more passes after the fact of this image you might not doing this process is a good way of creating a good seed for upgrading your image with upscaling and whatever um in fact I've had good um good results with having upscaling process that happens you know one you know every second sample or something and then doing a couple of face restoration processes in order to make sure that the details stay on the faces because and there's a habit of SD I'm just wiping the restored faces if you actually continue to sample too much after you've restored them so be aware that this process can be accidentally removed if you change the image too much after that you've restored them so it's better to use later on in your workflow but we're going to use this just like with one sampler for simplicity's sake and we're going to start with a custom node called the face restore with model node and this may be downloadable with the Comfort UI manager but you can also fire on civit AI is the face restore node so if you are having trouble finding it you can go to civitai and and you should be able to find it on there now um I am going to include jisan workflow of this setup as well just in case you're feeling lazy and want to download all the custom nodes but I do suggest you build these systems yourself because it'll be more easy to absorb the information if you actually take part in the process of building these systems so let's begin various restore models all generally will need an image input some Will May in the future at some point receive latent but most receive image so we sent an image and this one also requires an upscale model now most of your normal upscale models will be ultra mixes and Forex Ultra shirts and stuff like that but if you put a giftcan model into your upscaler thing you can use this to fix the faces of your model if you have the other ones on it's not going to work properly so you use this one use a gif game and you may need to download it from hugging face or civita or something like that pretty easy one to get hold of uh also if you have come for UI manager you can go to install models and you should be able to find one the many of the models that you might need will be in here if if you can't find them all right so now we're going to run this image and we'll just clone this because then we'll be the same size all right so Q prompt and you should see it's actually pretty quick to do this and you'll notice that it doesn't look like it's changed it much but it has sharpened up the eyes significantly it's sharpened up this a lot of the facial features so it's more about restoring kind of blurry facial features and stuff than about fixing deformities this one so it's useful if your faces are mostly fine or like a larger in the image this this method will work quite well but if it's like a a really deformed face it tends to not be particularly great so put that aside now let's try another one and this one is going to be a roof based model now roof is designed to replace faces with existing faces so we're going to go to image post-processing and we're going to use reactor fast face swap node and you may notice I'm not using the root node here and that's because some of these root nodes and other ones that use this process are broken now there are some difficulties with installing these particular root based nodes for conf UI and that's because um they actually require you to install um it's uh Visual Basic now you have to install Visual Basic on them so I suggest you go to the GitHub for these pages and install it will ask you to install a Visual Basic Studio or something and make sure that you include Python and C plus plus and then when you install the node it will actually install properly instead of throwing errors in the install process at which point this should all work properly all right so reference images we need reference images for this process and I'm going to use some people from Stargate and we're going to use Jack and the interesting thing about this is it can choose faces within the image which means that if you use more than one of these in the series you can also choose more in one face to fix and so the reference is where you choose which face in the image and it's counting from left to right which face you wish to use and we're going to use zero on both of these because there's only one face and the input faces in the is in the input image which face you wish to replace so this particular one the first one is going to be the leftmost image and the second one is going to be the rightmost image because we're going to switch this to one now if we clone this and we put this here key prompt you'll see that it's going to swap some faces for us which is what the Nerds are designed to do it's not the same as an image restore this one actually replaces faces so as you can see it's replaced these two people using these two Source images and you can see that they're not in the same pose they're not the same color that they're matched to the location they're in as well so it's quite a good way of adjusting faces you can also use it just as a seed for a thing so you could do this and then add a little bit more in a k sampler and it will it'll rewrite these faces just a little bit so that they don't resemble these characters anymore but they're still better faces than you might have with these weird um weird things under the mouth and funny noses and that kind of stuff all right and so now I'm going to show you oh yeah um if you put this on two it'll go to a third character you put it on and three it'll go to the next character and so on for input faces so you could if you just kept stacking this you could replace many many faces in an image or if you have multiple faces in one you could use one image in several inputs and select it using these all right now I'm going to show you um the third one which is impact pack so in backpack is a very very complicated set of nodes that is very daunting when first used because um basically they just have hundreds and hundreds of nodes it's hard to figure out what plugs into what and things just seem like so much more complicated than they need to be however the face detailer node is is incredibly good um you just have to make sure you plug it in properly which is the hard part so the most important things where you can mess this thing up are going to be in your bbox detector Sam model and second detector now the second two are optional but they'll generally improve the image that comes out at the end of this so we may as well use them because there's no reason not to you may only have access you may only see the mm detector on this by default but the one we want to use is actually the eurolytics the ultralytics detector provina because it is more generic and it works on the photographic faces not just Anime faces if you want anime faces you can use the mm detector because it will um it has it comes with a um with a different face from this one this one can also do hand models and stuff too but um yes so the mm detector contains an anime kind of model thing so with the Sam model do not use the Sam model loader because you are probably going to throw errors use the Sam loader instead because that'll load the right thing that you need now we're going to plug this into our boss there is another version of this called a face detailer pipe which is useful if you're using this node sets version of this they have a bus called a pipe which lets you plug in just like you plug in all this stuff at one end and then it sends the pipe is a single cable all the way across to wherever you need it to be so it's kind of like a way of combining all this stuff so it's useful I don't use it because um I find this a little bit more flexible and because I'm doing tutorials I want to show you people more stuff without too much confusion all right so we need this base image and now we've plugged this in I don't know what settings for this work best this is stuff that you'll have to figure out for yourselves I'm going to use this on default settings which seems to work pretty well and I'm going to clone this so that we have another one and we plug it in now these are going to be the little bits and pieces that it it pulls out in the process of what it does you can use these to do other stuff you could use the mask to do things you can use all these things to do different image editing techniques like cropping and pasting and all that kind of stuff if you want to use that for those kind of things all right now we generate again and it will generate some stuff in here now these nodes are pretty complicated they generally tend to take a while and it will generate one pass on each person that it finds as well so as you can see it's it's generated two faces which it's generated the uh them cropped as well if we go over here we can see that it has replaced the faces now they actually the interesting thing about this process is that it's a lot more similar to this original photo with similar resolution as opposed to the face replacement one which looks kind of like you've just pasted faces onto it by default but obviously it's not always going to 100 work so you may have to run this a couple of times to get this to to produce faces that you are going to actually be able to use so anyway as you can see that's a few different methods of doing um doing face restoring it works at Mid distances works at close distances as well allows you to replace faces with other people's faces and if you include a process after this where you run Samplers you can further improve these you can upscale them and so on um it's a very adaptive process and I suggest that anyone who's having issues with bringing out good images consider actually making use of face restoring nodes if you can get a hold of them all right so I'm going to include this just on with the video and uh what am I going to do next next tutorial is probably going to cover prompting specifically like all Liam's announcer prompting and syntax although it is a bit of a rabbit hole but we'll see we can do I might do embedding embedding in that as well I haven't actually used embedding much so I'm gonna have to learn how to do that first but hopefully what I learned can help you later foreign

Channel: Ferniclestix

Views: 15,142

Keywords:

Id: FShlpMxbU0E

Length: 16min 21sec (981 seconds)

Published: Sun Aug 06 2023