How to Generate REALISTIC AI Animations | AnimateDiff & ComfyUI Tutorial

in this tutorial we're going to learn how to create amazing animations using animate diff I will guide you through the installation process and share my settings along with useful tips and tricks so you can get really good results to work with anime div you first need to install either automatic 1111 or comy UI if you've seen my previous video you probably know that I'm starting to like comy UI more so I'll be using that in this video to get started you will want to check this amazing guide created by Inner refle Reflections on Save ey I will leave a link to the guide below and by the way I've had a really interesting conversation with in the reflections himself so stay tuned to hear some really useful insights from him to get started you will need to install three dependencies git is essential for downloading extensions and updates FF impa helps avoid certain errors and 7zip is used to extract archives after that look for the comi download link that matches your operating system if you're on Windows simply right click here and choose save as choose a destination and hit save once you've downloaded the file use seven zip to extracted open the extracted folder and navigate to confy UI custom nodes this is where we will download all the necessary extensions to do this select the entire file path type CMD and hit enter to open the command prompt box now go back to the guides and copy these four commands one by one the first command is used to download the main animate diff extension paste that over here and hit enter to download node repeat the same steps for the remaining commands to download the confi manager the advanced control net and video helper suit now I plan to add an upscaling node you can find this in the description box copy the link go back to the command box type get Space clone paste the link and hit enter to download make sure you have successfully downloaded all the extensions then go back to the main confi folder and open the Run Nvidia GPU file to launch comy UI on your browser now we still have a few more things to set up let's open the manager and then click on install custom nodes here search for auxiliary pre-processors and install this extension next search for fiz nodes and click on install once the download is complete you can close the installation window as well as the confi tab and the command window now let's talk about the AI models that can give your animations a certain look I've had great results with modules like dream shaper art universe and RPG artist tools to get the module right click on the download button and choose save as then go to the confy folder navigate to models checkpoints and place the models right there for slightly better results you can also download the vae model make sure you get the safe Anders version and place it under models inside the VA folder next in the installation guide you can find links to two different animation modules that are necessary when running anime diff go to confy UI custom nodes open the animate diff node folder moduls and save the modules over there the guide also explains how to download control net models you can go ahead and do that but these are only needed for video to video or video animation which I will cover in another tutorial so make sure you subscribe to my channel if you don't want to miss it all right now that we're done with the installation let's get to the fun part open comfyi this is the default workflow but of course we need to make some changes before running anime diff the really cool thing is inner Reflections has already shared a couple of workflows in the guide which will make your life a whole lot easier you know I'm glad I spent the effort to do this when you make your first stuff you want to cord it all to yourself but I realize that there's a lot more benefit to being you know open about it I mean that there' be a shame that you have something so powerful that is locked behind kind of it's a little bit janky right getting there but as soon as you get there and as soon as you have one workf just truly very creative people are suddenly using it for things you didn't even think about know that you know you're the first person guiding them in the right direction I think that's uh very rewarding it's fun to be the first right that's a crazy thing about AI you know you could be first person to make a guide to do this I think that's the most exciting about it right you like truly you can do something new let's download the workflow number five that is called text to video with prompt travel save it anywhere on your computer now all you need to do is drag and drop the Json file that you just downloaded into the comy interface to load the workflow this is already a great workflow to start with but I've made some tweaks and created my own version for today's video so go ahead and download it then then simply load it into comy UI and as you can see the difference here is that my workflow includes an upscaling node which will give you a much higher quality animation and now we can start tweaking the settings here you can choose the number of Animation frames you want to generate in this example I will go with 425 frames next you can change the width and height to your liking I found that using one of these Dimensions works best the highest I could go with without getting distorted results is 512x 768 so I'm using 512 X 768 it works great but anything higher than that it just generates multiple subjects it's a model issue generate lower and then upscale um however you want from there although it may be very low by today's standards but I will show you how to upscale and improve the quality in just a minute now it's time to choose a checkpoint model I will go with RPG artist tools for the motion module let's stick with a default v1.5 V2 The Prompt scheduler is actually quite simple to understand by scheduling different prompts we can achieve a change over time in our animation by default the AI will generate the initial prompt at frame zero and then at frame 30 it will generate the next prompt and so on you have the freedom to replace these default prompts with your own creative ideas to make your animation more personal if you're unsure or struggling with writing prompts I have a helpful video that you can check out before moving forward you can set the max frames to be the same as your frames value clear the default pretext and use this setting to automatically add keywords at the beginning of each prompt in my case I'm using it to describe things like camera angles to maintain consistency across prompts same goes for app text except that this is used to add keywords to the end of each prompt and I usually add prompts that are meant to enhance the overall quality of my output you can also use the negative prompts box to specify what you don't want to see in your animation now let's move on to the K sampler node you can play around with different values here but I found a combination that works really well set the steps to 30 the CFG scale determines how closely the AI should follow your prompt I'm going to set mine to 7.5 but you can go lower if you want to give the AI more creative freedom lastly set the sampler to Deep pmm 2m sde and the scheduler to Caris next you can set the seat to random if you want to get a noticeably different output each time this second Cas sampler node will work on upscaling your animation it's pretty straightforward and you can use the same settings from the previous K sampler node however I recommend setting the dooy strength to 0.5 or 0.6 to prevent drastically different results after upscaling you can increase the upscaling ratio but keep in mind that the process can become really slow for now I will leave mine at 1.5 as I plan to enhance this further later on using a much faster tool we're almost done with the settings and all that's left to do is set your frame rate in the anime diff combine node I'm going to set mine to 10 personally I prefer using the Webb format because it provides higher quality you can simply use the exact same settings in the combined upscape node and that's pretty much it at this point you can go ahead and click on Q prompt and the AI should start doing its magic the generative process might take a while depending on your settings but if you've done everything right and input the correct settings anime diff will generate a stunning animation for you if you really think you hit a true you know bug which you can people have the anime Discord developers they're pretty responsive what I personally do when I've run into errors I would go on Discord I use the search box and type in the eror there and usually there someone has asked about that already someone from the community would respond to them and there's a support Channel if you have a question look for that support Channel if you don't like the output you can simply open the queue cancel the current run and generate a new animation once you're happy with it let the upscaler run to get a much higher quality output which I'm about to upscale even higher to find the animation files go to confy UI and open the output folder and you will find all your animations there before you upscale the animation you will need to convert it to MP4 for that I'm going to use Easy GIF just select your animation file click upload and then start the conversion by clicking here once it's done click on save to download the mp4 file next import the video into topas video AI which is a very popular upscaling software let's set the upscaling ratio to two and then scroll down to change the upscaling model my go-to choices for AI animations are either emis or Iris if you want you can also change the file format and finally click on export the upscaled output will be saved right next to your animation and as you can see topas video AI did an incredible job at sharpening the animation and enhancing its details and now we have a video that looks 10 times better I'm really excited to see what you guys create with anime d so be sure to share your animations with me on Instagram or Discord if I were to encourage any of the people to make things is abstract stuff it's actually really good at and what it doesn't suffer from that we humans do is writers block you say make this even if it makes something really bad it will make something it's amazing that we have a tool that you can use free to do what you want it to [Music] do oh

Channel: MDMZ

Views: 28,533

Keywords: Ai video generator free, ai GIF, ai animation, ai animation generator, ai animation tutorial, ai animation video, ai art, ai tutorial, ai video, ai video animation, ai video generator, animated gif video, animated stable diffusion, animatediff, animatediff automatic1111, animatediff comfyui, animatediff text2video, animatediff tutorial, comfyui animatediff, generate ai animation, prompt travel, stable diffusion, stable diffusion tutorial, text to animation, text to video

Id: EC9bE7Fot78

Length: 11min 2sec (662 seconds)

Published: Sat Oct 14 2023