How to swap outfits in videos using Stable Diffusion with ComfyUI while keeping consistency

welcome to another pixela poets video in this video I will showcase how to change the outfit of a subject in a video without changing the pose or the background so let's Dive Right In Here is the GI up repository uh of the animated div outfit changer in it you will find three Json files a Spider-Man workflow a outfit two outfit one workflow an outfit to outfit to workflow and three reference MP4 files which you can upload to achieve similar results let's dive into the workflow starting with the Spiderman one um so I've uploaded already the rotating mannequin MP4 and in addition I have used the analog Madness uh Laura which is a Laura specifically for modeling and how to poses uh then I have used anime div for making the animations a bit smoother um but you could in in this case because you're using a uh all the frames anyway you could skip it and it would supering po but I like the smoothness that anima brings anyway then I have a control map model which is called outfit to outfit which you can find on CV Tai uh you can find it on GitHub uh but I don't think you can install it through the manager so find it there follow the instructions and just install it um if you have a uh strength of one it will as closely follow the input video um it will will change the outfit according to your prompt and if you uh reduce the strength you will give it more freedom to hallucinate um so then it's not exactly like the reference video uh but backgrounds will change more for example uh or the both might change differently so I prompted this case is say man wearing a Spiderman outfit and say man in this case is the tricker word so you could use the same man or the same woman uh and it will try to keep the same subjects uh and not hallucinate on this subject which is exactly what we want um so we have the same men wearing a Spider-Man outfit and uh as always we have our nons safe for work negative props so it doesn't generate NES and even though when comparing this tutorial it it did on the second demo that I will show you in a bit um so when this is applied it will use the cas sampler it will decode and it will create a video that is similar to this so that's when you press the generate button uh and you can see it perfectly created a Spider-Man suit on this particular character so let's now go to the second workflow let me open it right here so we've loaded up the outfit to outfit one workflow and as you can see we have this girl here uh as you notice it's a 16 by9 proportion and that is how I found a video online uh and I just downloaded it to demonstrate what we can do with it so that's what we have to work with I have disabled in this case the forced size so you can resize it but for example if I do a 151 12 by whatever then it would get really really small and that affects the quality uh so I've kept it as is um then we still using the analog Madness uh low right there we have the prom the same go wearing a black punk outfit with I cannot spell uh with a blue skirt and we had added naked to the um negative front because in my previous try it would generate a naked woman uh which is bad for YouTube so we are rendering with 24 frames uh again it will apply the outfit to outfit in this case I've chosen a strength of 0.9 to give a little bit more freedom uh it will look a little bit more natural and then we come to this results um you can see it is not 100% consistent um the outfit changes a little bit it changes color it does keep the pose more or less the same and um the subject pretty much looks the same and the background looks very similar so I'm happy about that um and you can play around with uh the seat number you can play around with with the with the strength to get a decided result different seat will probably uh generate uh slightly different results more in line with expectations and and that's basically how you do outfit to outfit um you could also use the uh open AI uh model that um is for keeping coherence between frames and unfortunately my video card does not have enough video memory to try it out just yet um so maybe I will do a fee on that in the future that will generate more consistent results but for now I cannot do that um so let's jump into the an last workflow so the last workflow is a pixelated poet dancing he's dancing on a stage uh with great dense moves I might add in this case we force the size to F 512 um you don't have to but I don't like waiting for R really really long time um the other settings are more or less the same so we do every frame we want the entire Circle and uh we have the same prompt as the previous example and in this case we say the same woman wearing a black sweater and a red bants and now the identity of this SPL poets is a secret um but I wanted to reinforce that it's a woman to also see if it would paint hairs despite we having uh wearing a hoodie there so let's continue the strength here I chose 0.65 um because there is a lot of movement in this video and I wanted a bit more creative result than um just sticking to the movement you can play around with it and see if you get different results in addition to the um analog Madness Laura I have added a clothing adjuster and it is a Aur that is supposed to remove cloths when there's a lot um I have not noticed a big effect for this yet uh maybe it has an effect maybe it doesn't um I was thought to put it at minus one uh you can also find this Laura at CV Tha um you can also rightclick and byp pass it uh it doesn't really matter much for the end result so then let's go to the video you will see here that the pixelated poets dancing uh it's pretty okay so it does somewhat confer to the uh prompt so it's the same woman wearing a black sweater which I can confirm this is a black sweater and a red pants and I can see red pants the feet do not stay in shape perfectly and there is a poltergeist moment with uh with the hat movement that is a little bit freaky but overall it is pretty nice um I did of course lower the strength quite a bit uh so if you would increase this it would hallucinate less uh but it's pretty great to see you can uh you can keep the consistency without using uh line art or without using Cy and you can create quite reliable results so that's the tutorial for today you can find everything in the give up repository and do check us out on YouTube we have a new music music for out it's called pixelated signal and it will hit Spotify the 8th of December thank you very much

Channel: Pixelated Poets

Views: 2,558

Keywords: comfyui, stable diffusion, outfit to outfit, clothing changer, outfit changer, animatediff, consistency

Id: Oa48SlGqdl8

Length: 10min 24sec (624 seconds)

Published: Mon Dec 04 2023