AnimateDiff Vid 2 Vid Prompt Travel

in this video we're going to go over how in comfyi using enimy diff how to use a vid to vid workflow to make something like this before we get started if you would like comment subscribe All That Jazz that'd be amazing without further Ado let's dive into it now if you don't have compy installed this tutorial is going to be a little hard to follow thankfully I already have a comfy UI installation video on my YouTube so be sure to go check that out and get comfy installed and then come on back here first things first we're going to need to get a workflow so if you're going to go to my website and you can go ahead and grab the image here save image as and I'll of course leave a link to the description below to that and then you're going to boot up comfy so what you're going to do is with that image if you're not aware you can just go ahead and drag and drop it into there and it will change everything that's going on so it'll it'll just load up that workflow now if you run into some red boxes that's okay what you're going to do in case you watched my uh installation video you're going to go to manager you go to install missing custom nodes it will bring these up and you'll click install and then you'll just go ahead and restart comfy but before you restart comfy now that we have all the nodes we need to get the animate diff motion modules so go ahead and download those and once they've been downloaded what you're going to do is you are going to go where your comfy is located you're going to go for me it's my e Drive and click on comfy UI I'm going to go to custom nodes then I'm going to go to animate diff evolved and then models and this is where those models are going to live keep in mind that those are different from motion lauras while M if still uses those they are not the same thing okay so now that you have those in there you can restart company and you're going to get a workflow that looks like this and the video we're going to use for this workflow that you saw I will link below and it's just this video from pixels of this guy dancing so you're going to go ahead and download that and then you're going to choose that file to upload this one that I'm using is a little bit uh cropped in so just be aware of that you might need to do that so we have our video here we have our width and height and these you can kind of adjust for your video generally for something vertical like this this works really well that's going to upscale it to the nearest exact then we're going to get some preview images and before I go any further something I like to do is I like to set this to 20 because it just lets me test things because we don't really have a preview option in comfy and then generally you know I won't skip SK the first frames if I like them and then I generally will always use select you know use a one select every nth so we went through that and it's going to go through this processor here for it's like open pose but DW is better but I am using the same control net for the open Poe control net and then uh you can see I have other options here if you want to add more in I just figured I would add some in if you want to for this video we don't have to use that but if it's not as coherent as you would like to be or something along those lines if you use soft Edge or line art those generally will help if you want to get these out of bypass mode you can do a contrl b or you can rightclick and click on bypass and it will undo that so that is one way you can get through with that and that we need to go back down here a little bit make the checkpoint I'm using is tune you but really any checkpoint will work and then I have just the standard vae now we have our prompt over here and this is for a batch prompt schedule you could just use a normal prompt box but I find I just use this more than anything anyway so what we're going to do is we have the pretext and app text and normally when you load a node like this they might not be there but if you want to put them in you can say okay cool let's convert you know a text to an input that would make this like like this for example if I want to bring these back in you can convert these to a widget now the reason we have these out here is just when they're in widget form it's hard to read everything that's in there because it's just like one line and it only you know gives you like 15 characters so it's hard to see so that's why we have the pretext and app text out here now what do these mean pre-text is basically going to place all of this before the main part of the prompt and apptext is going to replace what's ever in here after the end of The Prompt it just allows you to be a lot more organized if you're trying to go along a similar theme we have our Max frames you can really set this to be whatever for this portion 1,200 this totally worked for me you can also set it to 20 to match the amount you're testing with and we have our negative prompts down here as well now for our animate diff nodes so I will tell you right now if you do not have a context length of 16 and you try to go higher it's not going to work out well for you so just don't do it and anything like you might be able to get away with eight but if you can do 16 why not and then this is where those uh motion modules will live and I have a few different ones in here but we're just going to use the some this one that we downloaded the mmsd v15 version 2 now we have our case sampler and we have we skipped over the end code but it's it's pretty straightforward case sampler now you can see here we have our seed is outside of this and we could put it inside if we want for this tutorial I will show you how and so this is connected to the model everything here from the control Knight connected to the stack the prompts connects to here they all go together right here it brings it to the K sampler and say you know I don't want to have this out here right okay cool we can get rid of that you you can go ahead and delete it and then right click and then if you want to do you know convert C to a widget now it's within the case sampler and unless I have something I like I will put it on randomize and let's change this to 30 and CFG is fine at 10 let's make this you ancestral that's what I like and we have our vad vaed code and the save image and our animate diff combine at the very end here Now preview images can be useful I generally don't use them a lot and then save with save image is I don't really have a use for these images but I'm going to include them for the sake of this tutorial in case you do so now that we've gone over everything in there we're going to go ahead and run it and again when you see the green boxes and it's going through everything that's where it's at within what it's working with so the DW Pro pre-processor is going to take a little bit of time so just be aware of that it took around 20 seconds or so to do it's going through the case sampler and it's going to just kind of Chuck through this and I'll pop back in when it's done and then we're going to the vid code it's going to save our images and we have our animation so this is what our video looks like and right now it is at a frame rate of 12 you can change this to like say 15 that's what I generally like to do because then later if I want it's be at 30 frames per second or 60 I can interpolate that and that way I don't have to use as many frames when I'm generating it originally and I find that to be really useful secondly I always have the format in h.264 or MP4 and not a gif because the gifs only have 256 colors and so you'll get some funky artifacts if you're using it as a gif the other thing that we should be aware of is hey I generated this where does it live well if we go to where our compan is installed we click on comfy UI going to go to output and this is where all of this stuff is installed and if this is the video this is the PNG where that workflow is generated from then all of your images are going to be saved in a separate images folder and if I scroll up over here you have the capital I imagesimage and that's exactly where these are saved and that's going to be it for this video I hope you enjoyed it if you want to comment do that be amazing uh you know like comment subscribe and I will see you in the next one cuz I'm going to be doing a lot more of these see you

Channel: Russell Klimas

Views: 3,887

Keywords: comfyui animatediff vid 2 vid, comfy animatediff vid 2 vid, vid 2 vid, comfy tutorial, ai tutorial, stable diffusion tutorial, ai walkthrough, comfy animatediff vid 2 vid walkthrough

Id: IZ25nhK3Atw

Length: 9min 15sec (555 seconds)

Published: Fri Oct 27 2023