ComfyUI AnimateDiff IPAdapter ReActor Face Swap workflow #comfyui #stablediffusion

hi while we wait for Sora to release their models let's play a little with what we can do today with animated if today we will see how to change the face and clothes of a character we created in animation the workflow is relatively simple and of course you can find it in the description of the video let's build the workflow from scratch so you can see the whole process let's start with the load checkpoint I choose the cyber real istic load V and of course the animated if loader here I choose the V2 version model positive prompt in green a negative prompt in red in the positive prompt I write photo of a man looking into the camera standing on the beach sunset in background in the negative prompt text ugly horror we will connect everything to K sampler I also need an empty latent in the resolution I choose 512 in width and 768 in length and in batch size 16 this will be the length of the animation connect everything to vae decode and select output in video combine I changed the format to MP4 because I don't like the quality of the GIF I also set the seed I choose dpmp 2m in the sampler and the scheduler on Cara CFG 6.5 we press q and everything seems to work Beauty I really like the movement of the sea in the background will take care of dressing our guy let's start by showing the model what face we want to get in the final result for this we will use IP adapter we will start with apply IP adapter node we will connect the IP adapter model to it I choose plus face here we also need clip Vision adapted to 1.5 load image and upload our reference image and one more thing to remember prepare image for clip Vision we will put all the nodes in a group which will be a little more convenient we will connect the model and back to K sampler we'll decrease the weights a bit and add a bit of noise and press Q as you can see we get a figure a little closer to the image we upload but the result is very different from the starting point and the background is much less interesting now we will try to fix it with the help of a mask for the IP adapter in order to create the mask I bring another load image node to which I upload the frame of the previous animation we created before we added the IP adapter this is a very important point because we want to be as precise as possible in the position of the Mask or in other words where in the animation we want to get the effect of the IP adapter right button and select open in mask editor and draw the area where we want the effect of the IP adapter connect the mask to the IP adapter and press Q excellent you can see that the result is now much closer to our first animation both in terms of the background and the character and now we can continue before we deal with the face let's try to dress the character since he is standing on the beach I thought it might be appropriate to add a Hawaiian style shirt to him let's create the shirt first I Mark the nodes I need positive prompt K sampler vae decode and empty latent we will change the batch size to one we will write in the positive prompt product photo of Hawaii shirt we will add a save node and we will also connect the case sampler directly to the model because we want a single image and not an animation in this case put the K sampler of the animation on mute and press Q excellent now we have to connect the shirt to our animation and for that we will use another IP adapter I will copy all the nodes we will need that's actually all of them except for the clip Vision we will copy and paste the image of the shirt we created and in the image that we use for the mask we will delete the old mask and create a new mask for the area where we want to place the shirt I changed the model of the IP adapter to Plus you can also play a little with the weights and we have to connect everything of course you need to start the K sampler again and press Q as you can see we managed to dress our guy in a Hawaiian shirt but our background was a little damaged by this and it seems to me that it is related to the background that is in the picture of the shirt so we will try to fix it before we move on to the next step and let's get to know another very useful tool in comfy remove background before we continue we will connect a preview node just to see how this tool works and press Q excellent now we will connect this image to IP adapter and hope that it will solve the problem with the background excellent now that everything is working we have dressed our guy and arranged the background we can move on to the last step and for this we will use reactor to the input image we will connect the animation and to the source image we will connect the face that we use to the IP adapter and we will connect all of this to the combined video node in the face restore model I select gfp GN and press Q of course you can try without and with other models and see what works best for you and this is the result we got so we started with a basic animation with the help of the IP adapter we showed the model what fix face we want to get and with the face swap we get a result that is already very close to the face image we uploaded so I hope you learned and we will meet in the next lessons you are of course welcome to subscribe to the channel ask questions and like if you liked it and most importantly have fun bye

Channel: PixelEasel

Views: 2,153

Keywords:

Id: 9cmU-NbqrCo

Length: 6min 19sec (379 seconds)

Published: Sun Mar 10 2024