Generate AI Videos With AnimateDiff & ComfyUI | Text to Animation Tutorial

[Music] in this tutorial we're going to learn how to generate stunning AI animations using animate div all from text prompts I will walk you through the installation process and share essential settings to help you achieve excellent results I understand that many of you like to run stable diffusion using the automatic 1111 interface because it's simple and easy to use but in this video I decided to go with comy UI instead it might seem a little intimidating at first but trust me when I say that comy UI is set to outperform the a111 interface in many aspects and I promise you the installation process is a piece of cake I will leave a link below where you can download comy UI if you're a Windows user like me scroll down and look for the download link right click on it and choose save as to download you can choose to save your comy UI folder on your desktop or any location you prefer once you've selected the location and saved it a compressed file will be downloaded you can use windr to extract it in the same location and the extracted folder will be your main confy UI directory next thing we need to install is the confy UI manager which we can later use to download extensions and custom nodes you can find it on civit AI along with useful instructions and I will leave a link to it down below make sure you save the file in your comyy directory after downloading extract this and run the new batch file to install the confi manager if you have an Nvidia GPU you can use the Run Nvidia GPU batch file otherwise you can go with the CPU option however using this will greatly slow down the performance of comy UI and either way kyui will launch on your browser so before we get into more technical stuff if you're interested in diving into the world of AI and picking up some new skills you need to hear about the great deal that today's sponsor has for you skillshare is this awesome online learning community that gives you access to thousands of classes created by industry leaders specifically for Creative People Like Us With skillshare You Can level up pure AI art while also exploring a whole bunch of other topics like photography freelancing and more with the rise of new and improved AI tools skillshare offers classes on how to use these tools in a practical way so I personally went for this make AI work for you class by smsh mystery smsh is this cool designer and illustrator who has figured out how to use AI to come up with ideas and make his creative process much more efficient in this class he breaks down how to use chat GPT and Adobe Firefly to get inspiration references and discover precise techniques to level up your artwork whether you're looking to learn the basics of a new skill or start your own business skillshare classes like this are designed to take you from a beginner to an expert and hey if you're one of the first 500 people to sign up through the link below you will be awarded free access to the whole class library for an entire month and as an added bonus you will get a 40% off your first year of skillshare membership this is the default interface of confy UI it has similar features and settings as automatic 1111 but with the difference that everything is connected by nodes and you can change these connections to create your own workflows now that we have comi ready let's look at how to get anime diff to work I will guide you on how to install and use anime diff but I also recommend that you bookmark this GitHub page for future reference to explore different use cases and stay updated with new features to install anime diff open the confi manager and click on install custom nodes to find anime diff just do a quick search you will see two options and make sure you install the one by coing cic once the download is complete you will be asked to restart confy UI however before doing so there are two more things that we need to download real quick first you need to get the motion modules from this hugging face page that I will link below look for these three checkpoint files to download this v14 file for example right click on the download button and go to confy UI custom nodes open the animated folder go to models and save the file there and simply do the exact same thing for the remaining two modules you will also find other motion Laura files these new moduls are meant for controlling camera movement in anime div so let me know if you'd like to see another video on that finally head over to this page to download the vae file make sure you pick the safe tenser version right click here's download load go to confy UI models open the vae folder and save the file there and now we can restart confy UI close this tab and the terminal then run the batch file again to start comy UI in your browser the default workflow here is different from what we need for the animations however in the anime diff guides if you scroll down the first text to image sample you will find a workflow that is a really good starting point and instead of manually copying the settings over you can download the workflow screenshot into your computer and here's my favorite thing about comy UI you can simply drag and drop the image into your interface and voila the workflow will instantly load and be ready to go so let's go through the nodes here and see how you can customize this the first node allows you to select which AI modu to load you can either stick with the default option or choose a different one and I've been getting really good results with models like dream shaper and ghost mix let's go with the dream shaper model for this demo you can find it on CV it AI just right click on the download button and choose save file as go to confy UI modules open the checkpoint folder and hit save once the download is finished click on refresh and you'll now be able to load available models by clicking on this drop- down list let's load dream shaper next be sure to click on the Subscribe button so you don't miss any future uploads I plan to create more advanced tutorials on anime div in the near future here you can change the seed value if you switch it to randomize you will get a significantly different generation each time over here you can choose to change the output Dimension I'm going to set mine to Vertical 512x 768 I often go with a reasonably low resolution for faster processing but I will show you later how to quickly upscale and drastically improve the quality of your outputs the batch size setting is simply the number of frames you want to render right now you're limited to a maximum number of frames depending on which motion module you use the v14 module allows you to render a maximum of 24 frames however I found that I get better results with 16 frames now let's move on to what is probably the most important part of the generative process and that is prompting now if you're familiar with generative AI you probably know that writing a well-crafted diffusion prompt requires a certain level of skill but if you're new to this do not worry because there are many websites available where you can find really good good prompts and I've made an entire video about this so be sure to check it out for instance I'm using a prompt that I found on CR some websites also have negative prompts which you can use to avoid generating strange images you can find both prompts I'm using here in the description box and although this doesn't guarantee the exact same outcome it's a good way to start experimenting and learning how to write your own prompts moving on to the rest of the settings I usually set the steps to 30 generally speaking the more steps you use the better quality you will achieve the CFG value will determine how much the AI should follow your prompt and I find five to be a good value to start with set the sampler to DPM PP 2m sde and the scheduler to caras you can also adjust the frame rate with a total of 16 frames I will set my FPS to 8 so I can get a 2 second long animation and all that's left to do now is click on the Q prompt button to start generating and here comes the Moment of Truth the process is going to go through the nodes one by one and it might take a bit of time depending on your setup and settings and check this out seriously it's working but honestly I'm not super impressed with this result so let's just click on the Q prompt once again to see what else the AI can come up with now have a look guys I think I like this one much more also keep in mind that every animation is safely stored open the output folder where you will find each animation is saved along with a PNG image right besides it and here's the really cool part take this animation for example let's say you really like it and want to use it as a base for another output what you can do here is drag and drop the PNG file into comy UI to import the workflow and I personally love this feature I've been experimenting a lot with different models prompts and settings as a result I have compiled a collection of 100 animated gen Generations on my patreon page so feel free to use these as inspiration or a starting point to play around with the settings all right now let's look at two different methods to enhance the quality and resolution of your output the first option is to use one of the apps scaling workflows available on the guide bring the workflow into comi I've updated the inputs here with the same settings I used for the previous animation the only remaining change here is to choose an upscaling ratio let's scale it by two and click on Q prompt and you will end up with a way better output that's super detailed and high in quality the only downside is that it takes long to process a much quicker method that I like is using topas video AI just import the animation and in the video settings set the upscaling ratio to two then head over to the enhancement section for the AI model I found that TAA works great with animated videos you can play around with the settings here let's set the sharpen and reduce noise to 10 you can tweak the output settings as you want but I personally prefer using the h264 encoder and MP4 container then just click on export and the upscaled file will be saved right next to your animation and as you can see the upscaled version looks way better than the original now I want to hear your thoughts on comy UI drop a comment below and let me know if you're going to switch to comy UI in the future other than that I hope you found this video helpful stay creative and I'll catch you in the next video peace

Channel: MDMZ

Views: 27,635

Keywords: Text to animation tutorial, ai animation, ai animation tutorial, ai animation video, ai animation video generator free, ai video generation, ai video generator, ai video generator from text, animatediff automatic1111, animatediff comfyui, animatediff stable diffusion, animatediff tutorial, comfyUI, comfyui animatediff, comfyui animation, comfyui tutorial, comfyui video, generative ai, generative ai video, stable diffusion, stable diffusion tutorial, video to animation

Id: kmZ5S2X55fU

Length: 10min 34sec (634 seconds)

Published: Fri Oct 06 2023