How To Make GOOD AI Animations | ComfyUI + AnimateDiff + Detailer

today I will be showcasing a ridiculously powerful animate diff tutorial by Dr Lieutenant data one of the major driving forces behind the rapid and advanced developments happening with comfy UI I will show you the ins and outs of this workflow to allow you to create refine and even edit your animations before we get started make sure you have comfy UI and the comfy UI manager installed and up to date if you need help getting to this point watch this video here as I walk you through all of the steps now let's get started [Music] if you have comfy UI installed already and it's all up to date drag in the workflow file located in the description below you can load that in here if you're getting a bunch of red boxes and error messages it's as simple as going into the manager here then just install missing nodes then you should have a few nodes here the comfy UI impact pack The Comfy UI Inspire pack animate diff evolved and comfy UI video helper suite are all the nodes that you will need for this to work so if they don't show up here I will have a list of those in the description as well then just go up to install custom nodes then just find those nodes in here by searching in this box then just install and restart so now that we have all of that out of the way let's jump in to see how this workflow works so first off just load in your checkpoint here as always for SD 1.5 I will be using the dream shaper 8 model now if you don't have animate diff already and you just installed it you may need the motion models I will also link those below I recommend the version 2 for the SD 1.5 models which is this one here you can also download them here in the manager so if you go to manager install models you can go to animate diff and you'll have all these models here that will work for animate diff including the motion luras so I installed mine from hugging face so it's saying that I don't have this one installed but this is the one I'm using here or you can use the stabilized version for anime diff evolved there as well that's a good one too it doesn't really it's a little bit more stable but some sometimes it doesn't give you as much motion as I would like so that's why I'm sticking with the version two I will also link the hugging face page below if you don't want to install it through here so the default settings are fine I don't I haven't really touched this at all so you have your vae here so with the prompt obviously you input what you want and then it'll pipe it out throw it in the sampler and then from there it'll generate the animation with animate diff and then it'll jump down in here and then I'm not going to go into the technicalities of this if you want if you're interested in the science behind this you can go see Dr Lieutenant data's video where he explains the technicalities of it but essentially this is detecting the face in each frame and then just just kind of applying it with the motion model and then segmenting it out and turning it into a video here this one is a little bit it's kind of close because the original image or the original the original animation was already pretty good but you can see a lot of improvements on this like the makeup and stuff and the eyes have a little bit more detail to them so it's much better but as you start to play with this workflow a lot and start generating a lot of stuff you'll see that it gets really messy here and then once it gets you know segmented out and cleaned up that you'll see the power of this workflow then this final stage here is where you can actually input a mask to kind of change something in the animation itself so this is something that I've only played around with a couple times I haven't actually gotten a really good result but I will just kind of quickly go over how this part works so to Showcase this I'm just going to start from the beginning so I need a prompt and to get the prompt I'm going to use the Ty prompt generator cuz it is the quickest and easiest way to get a highquality prompt so for this I'm going going to do realistic woman in a white dress then it's just going to generate real quick cool then we're just going to copy that and actually paste this in the new negative prompt generator right there now we're just going to paste the positive prompt here go back grab the negative prompts and paste that cool and for this I'm going to use the same exact seed so let's queue it up just jumping here into the K sampler and then it's going to jump here boom I will cover on this preview image section up here but it's going to jump down here segment out all the frames I don't even know if that's the right term for it segmenting segs segs just sounds a bit odd so I'm just going to call it segmenting it's making segs it's ssing it's having segs immediately you can see a night day difference between the two this looks a lot better and between these there's no difference so the does this do well if you scroll up here so in this little corner up here we have a sampled image from the batch so this what we do with this we just copy it so we'll actually save this image then just bring it down here into the load image then just load it in now that it's in here we can imp paint by just right clicking then just open and mask editor so I'm just going to mask her lips like that should be good then save to node then just go down here and this is the prompt you want to run on top of this mask to tell the model what you want to change so tried this out before again didn't get the best result but I'm going to have her smile I want her to have you know nice clean teeth and we'll see if it works and then the default on the alpha is 255 so I'll just switch it back and then we'll just Q prompt since all the seeds are the same it'll just pick up right right here so Q prompt then it's just going to start here and of course since I'm showing this it's working perfectly I think it's because there's not that much movement in the in the one I was trying to do before her head was kind of moving differently so the Mas was actually being transported to different parts of her face because it was still in that same area on the canvas oras this one it's her mouth is generally in the same spot on the animation so I think that's why it's working a lot better but I'm glad it works so now you can see that it is actually really useful so yeah she goes from being low detailed to being you know much better detailed to being betterly detailed and smiling so see the power in this workflow and it's actually mind-blowing and I meant to cover this a couple weeks ago but has been insanely busy over here and there are a ton of new workflows tons of new new developments coming out in the AI space so stick around I got a lot more to show you and to finish this off I will show you how to even further improve this video by interpolating the frames and upscaling it so right now this is a 512 x 512 which is basically useless so I have it loaded into topaz video AI one of my favorite AI softwares so I'll show you why so we have this 512 x 512 at 8 8 FPS this is an 8 FPS animation so we're going to bump that up let's do 60 frames per second and then we're going to do and bump it up to a 1024x 1024 then it's going to run two different AI models the enhancement Ai and the frame interpolation AI so I'm just going to run the preview here now look at the difference buttery smooth 60 frames per second and it's 1024x 1024 all right that is the end of this workflow tutorial hopefully made sense I hope you were able to follow along and are now able to create your own animations using this method again a huge shout out to Dr Lieutenant data for this workflow in addition to all the work he does to make comfy UI what it is right now and will be in the future if you're running into any issues have any questions or just want to be part of a community looking to better themselves in all Realms make sure you check out the Tyrant Empire in the link below to join our private Community for absolutely free with that said my friend I am out of here until next time keep conquering

Channel: Ty The Tyrant

Views: 2,528

Keywords: tyrant, empire, tyrantempire, rampayj, ai, artificialintelligence, art, midjourney, stablediffusion, automatic1111, leonardoai, dalle, chatgpt, fitness, selfimprovement, selfhelp, learning, gaming, apexlegends, leagueoflegends, callofduty, dating, women, redpill, entreprenuer, freelance, freelancing, money, business, memes, streaming, tyrin, music, musicproduction, dj, howto, makemoney, america, usa, lamborghini, ferrari, bugatti, honda, funny, muscles, abs, sixpack, 6pack, diet, food, polymath, books, reading, god, comfyui

Id: L6M3qDHfwqk

Length: 7min 58sec (478 seconds)

Published: Tue Dec 12 2023