ComfyUI ReActor Face Swap Image & Video

hi everyone welcome to data leveling in today's video we'll be exploring how we can perform face swap using confy UI on both images and videos we will learn how to perform face swap on an image level and understand how it works we will then proceed to use the same technique on a video as the video is basically just many frames of images being put together we will require inside face for this note to work so you can watch my windows inside facee installation guide first to install it now for the dependencies we will be using reactor head over to your confy UI manager search for reactor and install the reactor node for confy UI next you will search for video and install the video helper Suite that will be useful for video loading and video combine you will then search for frame and install the cony UI frame interpolation note that will help us interpolate our frames to make the video look smoother with less flickering when merging the frames together lastly you will search for was note and install the was note site now we'll restart your confy UI to install the packages as for the models we will head over to the reactor GitHub repository click on installation open the tab for confy UI and install both the code forer and gfb gun models these model should go into your confy UI models forer face restore models folder the folder should be created when you restart confy UI all right let's get started so let's say we have two images here and we want to swap the face over from the bottom one to the top one let's use the reactor node to do this we can find it by right click add Noe reactor reactor fast face swap let's go through the parameters here the swap model is the one responsible for performing the face swap action currently we only have the in swapper 128 as that's the only open source model inside face has provided us with face detection here is to use to detect the bonding box of the faces in the image before performing the swap action you could use the retina face model or the YOLO V5 model both works well for face detection the input image is the face that we want to replace and the source image is the face that we want to use so because the open source face swap model is only able to generate 128 pixels you can see all our swaps are blurry therefore we require a face restoration model we have two options here code forer or gfp gun for the code former we have a code former weight option how this works is a smaller weight produces a higher quality image while a larger weight improves the Fidelity or what we call the degree of exactness of the face the face restore visibility is the clarity of the faces being swapped if you set it to one it will show the maximum Clarity of the faces for detect gender if we select female only the female faces in the image will be detect detected and perform the swap process as for phase index it is for selecting which phase you want to use in the case where there are multiple faces in an image if there is only one face in the image you'll just select zero the face index is counted from the left to the right and then from the top to the bottom remember that in programming the first item starts at index zero and not one we can also swap multiple faces in one pass for example in an image like this let's say we want to swap the mark zerber face to ronado and Elon mask face to Messi for our source face index it will be 0a 1 and for our input face index it would also be 0a 1 we could also swap both faces to Elon MK face by setting the source face index to 1 comma 1 remember to not put any spacing between the commers the console lock level just shows the locks that will be printed in our console level one is usually enough information now for video it is really just performing the same action to every frame in the video Let's create a load video Noe we go to add node video helper s load video upload this note is basically our video to image note and the parameters will decide how many frames we will get from this video Let's understand the parameters here Force rate is the number of frames per second if you set it to a value of zero you will just use the video's original frame rate four size is to change the image resolution if you set to disabled you will use the original image resolution if you want to keep the expect ratio you will use either custom height or custom width once you set either all the other will be down skilled or up skilled accordingly based on the scaling factor that is computed from the original image my stock video's frame rate is 25 frames per second and this video runs for 8 seconds this means that if we leave everything as default we'll get a total of around 200 frames depending on the milliseconds frame load cap is the maximum number of images we want to use if we use zero it will load all the image skip first frames is to set how many frames you want to skip and start counting the frame load cap from there select every end is to only select frames that are multiple of the number we set this means that if we set it to two we are dividing the frame rate by half once our image is swapped we will have to combine all of these frames back into a video we will use a video combine note here from the video helper site select video combine the frame rate here is for the final output so if let's say the output images coming out is around 200 frames by setting this value to 25 we will get back a 8sec video but if let's say we set it to 50 then we'll get a fast forward video that only lasts for 4 seconds for the form format depending on what you choose there are different options if you're using h264 MP4 pixel format is how the pixel data is stored the one with P10 is for 10 bits and the original one is for 8 Bits the CRF value will determine the quality and file size lower value correspond to higher quality and greater file size note that if you are using av1 you require to install the FFM pack on your device and set it to pu Loop count is how many additional times the video should repeat and ping pong is to play back the video In Reverse for the face swap process we do not have to worry about vrm issues as it is running iteratively for each swap that is happening but this also means that the process might take a long time if you are working with long videos now because we are performing a face swap action we do not really have much control over the final output making our video look something like this where there is a lot of pixelation why this happen is because as the head is rotating different frames of the head movement are going into the face swap process this might lead to some inconsistency from the current frame and the next frame leading to the flickering that we are seeing now so what we can do is to reduce the number of frames that will go through the face swap action by let's say selecting every fif frame in the video this one works for my case as there are not a lot of of big motions in this video but if your video has a lot of big movement you should reduce the value to maybe two or three you will then use a frame interpolation method to synthesize the frames and make the video smoother go to add notes confy youi frame interpolation vfi and then film vfi so film or frame interpolation for large motion is developed by Google what this does is that given two Steel images it gener generates out the in between frames to make it blend smoothly since we reduce the original frames by five times we will multiply it back by five times this also speeds up the entire workflow as the face swap is the one that usually takes the most amount of time another thing we can do is to increase the frame interpolation multiplier let's say we double up to 10 we then have to double up the video combined frame rate to 50 this will ensure that the video will be kept in the same duration as the original do take note that the frame interpolation requires a lot of vram so if you're having issues with it you may want to add a rebatch image note in between here are the results comparison now what if we have multiple faces that we want to swap in a video and some faces are changing their positions over time let's take this video as an example our Target is to swap the first two men in the video with our Tech Giant Founders if we do it normally we will get something like this if you look at this point over here when the guy at the outer left moves out of the frame the index for that frame changes and the swap fails so in order to fix that we can first save all the frames so we can see which frame fres are being affected for this I'll be using the was node Suite we go to add node was Suite IO image save the main reason for using this instead of the building save image is because this one helps to create a folder within the output folder therefore not clashing with the image index in the default output folder we will set everything to false here except for preview let's see where the index change happened we can see that around the 217 frame that's where the change begin once the images are saved we will go to the video help us s select load images path so there are two ways we can do this the first way is to duplicate the load image path Noe and then we Skip and select the frames so for the first load image note we will set it to load image cap until 216 we will do it as per usual using the same reactor Noe the second load image note we will set it to skip the first 216 and then let it run until the end of the video we will link it to a new reactor note we can duplicate it finally when we are done with the face swap we have to merge the images back into a single batch and send it for frame interpolation go to add note video helper s image merch image badge and for the second one we have to change the source pH index to one and the index pH index to be zero now we change the path to where the images were saved previously the other way is to use the Split Image batch note go to add note video helper site image Split Image batch we will set the frame Lo cap to be zero to load up all the image in the folder and then we set the splitting index to be 216 batch a will be the first split from 0 to 25 and batch B will be from 216 onwards and here is our final [Music] result if you learn something from this video do help to leave a like And subscribe for more content like this it will really help the channel grows and serves as a motivation for me as well if you face any difficulties following the videos do also leave a comment and I will try my best to help you and remember don't stop leveling up

Channel: Data Leveling

Views: 2,162

Keywords: deepfake, img2img, vid2img, img2vid, stable diffusion, deepfake video

Id: PQLttrd-Epc

Length: 12min 16sec (736 seconds)

Published: Wed Feb 28 2024