FREE ComfyUI Animation Workflow With IPAdapter!

animated diff used to be limited to very short videos but no longer now you can make AI generated videos of any length and even do morphing as well all thanks to batch prompting remember the old days of VQ gun clip where you had different prompts at different times well even if you don't remember the video playing right now should give you an idea of what you can do but how can I do this myself and is it free I hear you shouting at the screen well dear nerd let me fill you with the Arcane knowledge is required for the princely sum of nothing if you're ready to begin here we can see from this Fork of the animate diff software all we need is it what people don't like editing Json files and they'd rather just use a weapon well why don't we do that instead yes hello and welcome to More nerdy rodent geekery where today I guide you in the most minute detail through this single workflow so that you do can make wonderful animations like the one you see on the screen now using the power of stable diffusion and comfort UI you'll easily be able to create trending masterpieces to woo your friends and thus become irresistible for this you're going to need comfy UI up and running already so if you've not done that as yet and you need help with it then do be sure to watch my beginner's guide video in it you'll find loads of really useful entry-level information and that should be your first stop before diving into this video if you've never ever used comfy UI before that way you'll have a great starting point with knowledge about simple comfy UI workflows where models go and a variety of other basic hints and tips you will need a somewhat recent graphics card I'm afraid if you've got something like an Nvidia 1060 from 2016 then AI animation may be a bit beyond the power of your card right let's get on to the setup now links for all of these things can be found down in the video description but your primary site the place you really want to go where everything is all gathered in one place is the very comfy nerd website here you can see a table at the top has various resources and also just down below you've got all the workflows if you start by dragging and dropping a workflow image from the very comfy nerd website into your comfy UI interface then you'll either be good to go already or you will see some red nodes a red node means a custom node will need to be installed as you've watched all of my previous videos you will of course have comfy UI manager installed already and if that's the case you can just click on manager and then click install missing custom nodes you'll also want to update all then close that and stop and restart comfy UI it may take a moment to complete but once it's done you'll have all the new nodes ready to go just in case you forgot to watch my other comfy UI videos in which I go into how cool the manager is the link for it is also on the very comfy nerd web page okay so now you've got the prompt travel with iPad to workflow loaded you've installed any missing custom nodes you've updated everything and you've restarted meaning if you now go down to the little menu and press the refresh button there it is refresh then that will update all of the options in the available loaders to match the models on your own computer for example if we have a look at these checkpoints there you can see a list and the list on your own computer will likely be very different to the one on mine being an avid watcher of the nerdy rodent YouTube channel you will of course already know the default locations for these various models is comfy UI stroke models and that there are loads of sub directories in there with the names to match each of the model types for example here you can see one called checkpoints and that's where the checkpoints go there's also one for clip vision control net vae and so on some custom nodes however have their own models directory if we have a look at the custom nodes and take a look at the IP adapter one you can see there it also has its own custom models directory now this may seem confusing at first but don't worry as I will specify exactly when it isn't the standard default location that everything usually goes in normally right let's go through all the models that you're going to need for this as the chances are you may be missing a couple okay so there's quite a few models for this one each of them should be specified in these purple loaders here in the loaders section then the first one you'll need is a stable diffusion 1.5 checkbox point along with a vaa if your model has a vae baked in then don't worry I've got a little router node up the top there so you could just connect that into the route and that will use the built-in vae or of course Like It Is by default you can have a separate one there next is the animate diff and animate diff Laura loader now this is one of those custom nodes which does have its own models directory you'll need at least one animate diff motion model as well as this model checkpoint here now if you're going to be using the motion loras you do need a version 2 checkpoint if we have a look at the web page for this extension it gives us all of that information here so download motion modules you will need at least one so this one with underscore V2 you can use with the motion loras as well as these fine tunes as well the 0.5 and the 0.75 pth as it states there as well place the models in custom nodes comfy UI animate diff involved models so if I show you over in my directory we've got our custom nodes comfy UI animate diff evolved and then we've got a models directory there which has all of those various models in now just to give you an example of how to download one of these files into the directories that they specify so here I'm going to download emotion Laura from the hugging face side so let's open that link in a new tab that will take me over to hugging face now it looks like there's nothing there but if we click on files and versions you'll see all of the different files there and let's pick one we'll have this Laura tilt up if you click on download file then you'll get another window up here and you want to download that into your custom nodes comfy UI animate diff involved motion Laura directory so you just click save there and that will download the file straight into the correct place the clip vision and IP adapter models are next now this is a mixed location one with the image encoder going in the default clip Vision directory but this IP adapter plus the sd15 one there that has its own directory the IP adapter custom node so let's go and have a look at that so we'll close that one down that's the old one and we want the IP adapter models directory there it is and there we have the matching file IP adapter plus sd15 so you can see there the names match up exactly if you rename the files that will rename it in there as well you may once again have to click refresh and pick the model that you need and the final thing to load down here which is kind of optional is an upscaling model once again links from the very comfy nerd website will take you over to this list of millions of upscalers all right there's a few hundred in there but do download from the categories available the type of upscaler that best suits your own requirements okay with the setup complete you're ready to start making animations most of the previous things you'll hardly ever change once you have set them perhaps with the exception of those motion Laura's as you may want to change the directions you can chain them as well so if you just go over here if you want a different motion Laura as well as the one you've got there you can just drag over there Ade animate diff load Laura there you go you're using pan down and zoom in but I'll just delete that one for now so you can have multiple motion lauras there as well which is very handy when making a new animation the things you'll want to pay close attention to each time are the weights section down here the load images directory those are the images for your IP adapter you've also got the resolution and frames and of course the prompting section above if we start with having a look at the weights section this section controls the balance between your IP adapter and your prompts for more information on IP adapter just take a look at my previous video about the instant loras but essentially IP adapter controls the generation without the use of having to have prompts let's have a look at what's going on here if you have a strong IP adapter combined with low prompt weights then you'll get a video like this where your prompts barely impact the video at all at around 0.5 strength on each you'll get a rather interesting blend my personal favorite is to blend them as you get really weird stuff going on and finally with the IP adapter strength down at zero you'll only get what you prompt which is basically having the weirdness setting on maximum talking about prompting you'll see this isn't quite like your usual prompt text as it's using a batch prompt schedule each of the prompts given there come into effect based on the frame number given just before it so there you can see Zero I've got a rodent eating pasta 24 it goes to seeds then beans ice cream and finally ending up with some cheese there are a couple of extra boxes here you can prepend and the append text as well if you like and underneath you've got the negative prompt box now do remember to follow this prompt format exactly so you've got a comma at the end of each of those but not on the last one if you do get it wrong so let's put a comma in there and try and cue this prompt up then it will tell you that you've done it wrong error occurred when executing batch prompt so there it is executed property naming closing double quotes yet no you've you've put a comma on the last line so remember to follow that format exactly to avoid such an error okay it may seem like you're taking in a lot at first but we have nearly finished the final section down here is the upscaling group now if you don't want to do upscaling at all you can just right click on that group and set group nodes to never that will gray them out and then you don't load the upscaler and you don't do the obscaling perhaps handy for slightly lower vram cards but it does give you a larger resolution video at the end of it and there you have it videos of any length with consistent characters objects Styles and such via the IP adapter which can also morph and change along the way as you like all for free from the very nerdy rodent web page I hope you enjoyed the video and to see you on the next one

Channel: Nerdy Rodent

Views: 18,674

Keywords: nerdy rodent, animatediff, ai generated videos, animatediff ipadapter, how to make ai generated videos, ai video guide, ai generated video tutorial, help making ai generated videos, generative ai videos, Stable diffusion, ComfyUI, Ai video, Ai video Howto, Make ai videos, Ai powered gif, Ai generated video, Ai video guide, Ai video tutorial, Free ComfyUI workflows, Stable diffusion animation tutorial

Id: 6A3a0QNPhIs

Length: 13min 45sec (825 seconds)

Published: Wed Oct 04 2023