ComfyUI: Vid To Vid AnimateDIFF Workflow Part 2 | Inpaint AnimateDiff

this is our last vid to vid workflow that I explained in the last video and if you watched my last video you must remember that I edit IP adapter to this workflow which allowed us to load an image as a style reference I have made some additional changes to this by adding an additional detailer module in this video I will show you what changes I have made and how it can be useful but first we will see which notes we need to install for this to work go to the manager and install these two impact and Inspire pack by Dr Latin Tate both these notes are created by the same person who created comfy manager and these two notes are a must to install this time I'm going to load an open post dance animation that is available at civit AI this animation will only give us a dance poses and we are taking the style and composition from the IP adapter image as explained in the previous video okay I have loaded this dance animation here I will use lowest resolution supported 512 by 512 now since this animation is already looking smooth I will probably not need this rif interpolation note so I will just bypass it right click and bypass now that we're using open pose animation directly we have to switch our control net model to work with open [Music] pose the good news is we won't need a pre-processor [Music] anymore so let's set the IP adapter this time weight is slightly higher because I have loaded direct open pose animation and the whole composition is going to come from [Music] here here I have used a node called change image batch size which is from the Inspire pack note that we will get two frames from the video which will be needed later here in the detailer group okay let's run this here are the frames of our animation [Music] okay not bad is not perfect but it can be improved let's see the face swap then we'll address other issues this is our face swap not bad the eye movement is looking a bit weird but that's how AI animation is for now however the focus of this video is not on that matter let's move [Music] ahead copy one of these frames [Music] activate this group again and paste it in the load image let's suppose we have to change one thing in this animation that we don't like for example we want this girl to be wearing jeans in this animation so we write the text prompt girl wearing jeans by right clicking on this image we open the mask editor and draw a mask of the area we want to change now since this is an animation over this area we want to mask this entire area so basically what we are doing here we are converting this mask into seg with mask to seg node and running it through detailer node for animated if and segs paste node to paste it on the original video fall which I got from the face swap node okay I can see in the preview that it's working out in the mask there here okay so this is our result you can see as we had prompted it has made the girl wear jeans undoubtedly this method holds significant potential for effectively altering various elements within our animation processes let's do some more tests for this I will bypass this group for now right click and bypass group nodes let's use all the frames of the animation this time if you don't want to set frame limit and load all the frames just leave this image's limit to zero I'm setting it to [Music] 48 set the same batch size in the empty latent [Music] image let's attempt to reduce the influence of the YP adapter by lowering its weight and amplify The Noise by doing so I anticipate the model to exhibit greater creativity in its generated output this [Music] time I'm going to test the anime diff V3 model now let's open the manager to install the V3 model it's already installed in the system this will help me explain why I still prefer mm stabilized High however in certain situations using V3 might be better since it claims to provide smoother animations [Music] so let's load the animated Dev V3 model okay let's run this okay let's see what's wrong here I think there's a problem with the anime diff V3 model file that I have installed I will need to reinstall the file from the manager let's see if I can reinstall from the manager itself no I need to delete the model file in the folder first let's locate the model file in custom nodes anime def evolved models I'm deleting this file here let's go back to the manager to install the file again okay I have loaded the V3 model again let's try this what quality anima if V3 model is able to to generate all right the outcomes are better the animation now appears smoother and the overall quality is greatly improved with animate diff model version 3 however it seems a bit more like cartoonish and less like realistic compared to using mm stabilized high I'll keep trying out this V3 version in the future and hopefully find the right settings for it to generate more realistic animations but that's for another video let's get back to the detail of module we are working on in this video copy one of these frames that are generated here activate the detailer group right click and set group notes to always paste the image in the load image here and right click open mask editor I am masking the area where I would like to see changes this time I will try changing the Shir so I'm rewriting The Prompt as girl wearing white shirt [Music] okay it actually has put a white shirt inside the upper garment anyways you got the idea how to do this and the download links are in the description give it a try let me know in the comments below how it's going I will put out new videos about my experience with comfy UI and AI image generation so stay tuned and thanks for watching And subscribe if you haven't yet

Channel: Zuhaib R

Views: 2,468

Keywords: comfyui, ipadapter, inpainting, aiart, aianimation, AIINFLUENCER, AI INFLUENCER

Id: 1djV0xKq-0E

Length: 7min 24sec (444 seconds)

Published: Fri Jan 19 2024