Better Face Swap = FaceDetailer + InstantID + IP-Adapter (ComfyUI Tutorial)

hey everyone my name is way welcome to my Channel today we are diving into a pretty common issue with instant ID in confy UI especially if you try to do a face swap here's the thing it tends to keep the same composition as your reference image so let's say you upload a head for your reference and you are hoping for a full body image from your prompt what you end up with yep still a head short and it doesn't matter if your reference is a half body short either ask for a full body and you are still getting just a half but don't worry I've got a fix for this I've come up with a workflow that lets us swap the face in the photo with any reference image we want so let's get into how you can do this too and make the most other the instant IDE let's jump right into how I tackle face swapping with some Cool Tools first up I use sdxl to generate a crisp portrait photo next I feed a couple reference images into instant ID and IP adapter these tools are great because they can help us pull out all those detailed facial features that we need for a solid swap now for the background ground of our face swap you've got options you can use an image from M Journey or even a photo you've taken yourself it's all about what fits your vision I've got the whole workflow Linked In the description below so make sure to check that out and hey if you're curious about similar techniques I'm also explod face swapping using instant ID and IP adapter in web UI Fork it's pretty neat and yes there's a link to that video down below too all right let's dive into creating the workflow that will generate the photos we need for our F swaps using sdxl first things first you want to make sure you have efficiency nodes installed in your conf manager got that great next step we are going to load up SD XLS effici C loader then we pair this with the key sampler for sdxl which also relies on those efficiency nodes now let's connect these notes together pick a checkpoint write out a prompt and generate a photo once our image is ready it's time to get into the actual face swapping we use a note called face detailer for this or before you load face detailer make sure the impact pack node package is installed in your config manager this note is super handy as it's F to paint and correct disfigured faces plus it automatically recognizes the face area which means no more drawing face mask by hand you also need to add a note that detects and the segments of the face region let's get that set up as well now let's connect all these nodes if you're looking for more details on face detailer I've got another video where I use Laura and the face detailer for face changes in confi I'll drop that link in the description too finally hit the Q promp button to make sure all the the node connections are smooth and there are no issues all right everything's looking good now let's bring in instant ID to take this up a notch before we dive in make sure you've got the instant ID node divided by cubic installed in your confy UI manager just search for instant ID and select the apply instant ID node we also need the instant ID model which you can drag out from the instant ID input Port plus this setup requires an inside face model to recognize visal features and since instant ID Works alongside control net we need to track out a control net model known as well next connect the image input port to the note that loads your image and then upload a reference image for this sto I'm using an AI influencers image from Instagram you find both the model and the prom can be directed from the sdxl 2on node then simply connect the output port on the right to the input Port of f deter and just like that we ready ready need to change the face all right we swapped the face but it seems a bit overfitted let's tck it a bit turn down the CFG slightly and increase the step count to give us more refinement phrases now let's generate another image and see the difference now if we compare here the new image with our reference face image you you might notice it's not quite as similar as we'd like this is where the IP adapter can come into play to boost the resemblance in our face swap let's add that next go ahead and double click and type unified to find the IP adapter that includes face ID if it's not showing up for you you might need to update your confi and the confi instant ID node we choose the face ID plus V2 preset for the IP adapter this note is fantastic because it automatically configures the best version of face ID for us whether that's SD 1.5 or sdxl once we have the model loaded into our IP adapter we need a main IP adapter node connect this to the unified loader the model input of the unified loader should connect to the model output of the sdxl tuo then take the model output of the IP adapter and Link it to the model input of the instant ID for the image input this can either connect back to the reference image using in instant ID or another image loading node all right setup is complete let's generate this new image and see how our adjustment will improve the face swap all right let's take a look at how our adjusted image compares to the original reference head shot it looks like we are getting closer in similarity but there's still a bit of overfitting happening let's tweak our setup again we head back to the ADP adapter node L the ways of touch and reduce the number of ending steps then let's generate another image it seems we might have F the setting down too much for this time let's Notch the parameters of the IP adapter backup bit and generate another image now this looks better the similarity has improved but if it's still not quite right for you remember you can always adjust the weights in both instant ID and ipf dat to F tune further noticing any small issues like maybe something off with the characters ears or forehead we can handle this with some imp painting also try tweaking the settings inside face detailer if you are finding that the face swap with instant ID and the IP adapter isn't quite hitting the mark on similarity there's another strategy you might want to try training Aur specifically for your project I've actually done a detailed tutorial on how to do this and it can really boost the likeness in your face swaps and be sure to in the link to that tutorial in the description below once you've got the Laura train you can integrate it into confi to enhance your face changes and you are likely see a noticeable Improvement in similarity I also done this tutorial before and I'll put the link to it below as well that's all for this tutorial thanks so much for watching if you liked this video don't forget to hit hit that like button and follow for more updates catch you in the next one

Channel: My AI Force

Views: 11,561

Keywords: ai art, stable diffusion, generative art, stability ai, stable diffusion tutorial, comfyui, instantid, efficiency nodes, faceswapping, facedetailer, ipadapter

Id: TTk00FLc2kY

Length: 10min 35sec (635 seconds)

Published: Thu May 02 2024