ComfyUI + Face Swap Ultimate Workflow for Realistic Results

hello everyone welcome to House of dim one of my first videos was a workflow for face swap it was okay it wasn't bad it was better than what we could do at the time but I think this workflow is by far producing the best results for me if I provide some value and you want to pay it forward please visit cure C md.org and donate to them let's get into the workflow here's the entire workflow here and we will start over on the left for this workflow I wanted to do something a little more realistic so I'm using anal L Madness we have our positive and negative prompts and the prompt I have is photo of a woman facing the camera shot from waist up black dress black hair bangs serious piercing blue eyes and dark background this is the empty latent I'm using RG3 to hang on the seat I already have a seed in there and we will go ahead and cue it so we can see the results as we go through moving over first case sampler I don't even preview it cuz I don't care second K sampler and this is what we're starting with and the face we're starting with and the piercing blue eyes part is important because I want to show you what uh normally happens with the eyes with a face replace and I'm using reactor to start with so from here there's a few different directions we can go let's go up first well let's go over here here's the face replace and it's it's not bad I mean it's good but when you zoom in you'll see it's the face is soft and there's a little pixelation going on here so let's go up from here well I'm using my wife of course for the face as as I always do she's my favorite model go up here we have a simple face detailer node and this is just to get um well I want to add a little bit of detail to the the original face that's there because we want to I'll show you why here in a minute and it's also to get the mask of where the face is at here we convert The Mask to an image we blur the mask because we want it we want it to be soft when we cut and paste it if it's if we don't blur it you can clearly see the square where we cut it and paste and since we're since we're blurring The Mask it it actually makes the mask bigger and so if if we cut and paste just the blur the face is going to be a little bit larger than what it should have been so we shrink the we shrink it back down using a mask morphology node and this node's pretty neat right now it's on the road which shrinks it but you can there's different settings here you can dilay which enlarges it but since we blurred by 10 we're going to shrink by 10 that seems to work pretty well if we look over here this is the mask that's been cut and this is what the mask looks like you know the the black and white and I'm sending that face through ldsr upscale and you could send the entire image through it and you'd get pretty good results but it takes so much time and depending on your GPU so much vram this way you know we're just sitting a little spot this spot right here through the upscale and this is the results of that and it's pretty good like if you zoom in you can see some skin texture here and normally this is completely lost with the face replace reactor face replace node the only issue I sometimes have is if you look at the eyes here they're um yeah they're just kind of they're just kind of different and the biggest issue is in the prompt I put piercing blue eyes and that's completely lost here even though the reference photo even has she has blue eyes and it's just completely lost so how do we fix that so for that part we go down here to this node and I'm sting it through a face detailer but I'm using um eyes for the Box detector for the face detailer I'm using impact pack kind of hard to say three times real fast and under simple face detailer for the eyes detector you'll need to download it and I'll put a link down in the description but you'll download it here and you will put it inside comi models ultral listic vbx right there we take the mask of those eyes convert into image upscale by two just so it matches our final result we're going to blur it also you know again to feather it kind of make it softer I'm going to shrink it in because I want to try to just get as much of the eyes as I can and you can play with this depending on the image you have you might want to shrink in more or expand it the road is shrink dilate is expand yeah just depending on what you have you might want to change that and you'll see what the mask looks like here I'm sending it all through ultimate SD up scale just because I like the extra detail it usually adds we're going to paste the face the image to paste we're going to paste the face on top of the the base image so we're pasting the ldsr upscale face onto the ultimate SD upscale and we're p and we're using this mask up here to control it and this is what we get here pretty good results right here but we're you know we don't have the eyes part yet so down here we mix the original image with our face replace image and we're using the ey mask so we just get a little bit of the eyes back come over here and this is what we got you'll see the nice blue eyes there and it's still it's still her likeness let me see if I can get them right next to each other see it's still her likeness there so really good results here it's nice you know the skin details here the eyes are blue again which they need to be and that was the biggest problem I had one of the biggest problems I mean there's a few with reactor face swap is her my wife has pretty blue eyes but every time I ran it it would turn a brown so that fixes that issue I'll reset the seat over here and we'll run another example I don't really like that one yeah we'll try that one again here's the face replace and just just real soft right here and I think that's what ldsr fixes we've cropped out just the face and this is the the bit that's going to take you know depending on your GPU is going to take a little bit the time but it's it's way faster running on just that little spot than it is the entire photo here's the upscale and you see nice nice skin detail there and again the eyes her eyes are not brown it's ran and already got the eye mask [Music] out and here's our [Music] result you see it's it's the exact same eyes that we got from the the prom not not quite as piercing blue as I like on the other ones but that's you know sometimes you don't get exactly what you want I'm going to run it one more time and we'll do something less realistic just in case somebody wants to see how that works my favorite I think is Dark Sun I'm just going to keep the [Music] prompt okay that looks [Music] cool there's a face replace there's the cropped out face it's going through ldsr again there's the results there this is the face we started with I'm hoping we get those pretty blue eyes back in the face [Music] swap and they're there you'll see the blue eyes are missing and and we got them back here and they're nice and I look a little more detailed too so that's my workflow for face swapping with comfi as usual links to everything will be in the description if you have any issues please post them and I will try to help you thank you for watching hope you have a great week

Channel: House of Dim

Views: 7,272

Keywords: comfyui, stable diffusion, ai art, tutorial, face replacement, face swap, ldsr, upscale, ldsr upscale, realistic, realism

Id: AdUhPHskeNQ

Length: 8min 14sec (494 seconds)

Published: Wed Jan 31 2024