Animatediff in ComfyUI Beginner's Guide to AI Animation

[Music] in this video we will be using animate diff in comfy UI to recreate some of this animations to achieve the final look we are going to build this entire node from the top I will clear this to get started our first requirement is to download a workflow for animated inside GitHub we are going to use a workflow by coyn Dink we are going to use the workflow here Tex to image 32 frames animation right click and save this to anywhere on your drive we are going to reload the workflow by hitting the load tab here locate my workflow I'll click on open opening up the wordlow we can see we are missing a few notes by seeing this red boxes and we can fix this by using the UI manager we come to manager we go to install missing custom nodes this will be the missing node for the workflow install this by checking this box clicking install we can see installing animates if Evol to apply we need to restart comy UI and refresh the browser I'm also going to update comy UI uh once this is done I'll close this I'll also close this we have to close this again uh back in the directory I'm going to run this Nvidia GPU again to restart comfy UI everything has been installed correctly I'm just going to hit Q prompt to see what we get from the default settings we can see we are still missing some inputs from the specific nodes in comfy UI we don't don't have the models for animate div and also the checkpoint which was used is also not in my checkpoint list what we are going to go for is the dream like checkpoint if we hit Q prompt once again uh we are missing the model for animate loader to fix this we can do this in two ways option one can come to the UI manager install models we can search the model we are missing here once I put this in here as in the specific model name click on search we can find it here I click install the patient want to click install because these files are a bit heavy installing from the UI manager uh we click on close we click on close here option two um on the hugging pH page here we can see a list of all the models available here this mm versions are the models for animated diff and this V2 Laura are the Laura motion models on this page we can see the Laura motion models which helps us to animates the camera and animates the which are the same checkpoints we have here for the Laura motions we are just going to hit the download here and the comfy UI we come to custom nodes once we have animated di installed earlier on from the nodes now this is where we are supposed to put in both the models and also the motion lers once I have my models downloaded I'm going to paste them here we can see we only have one checkpoint here because that was what we installed from the manager UI earlier on uh we go back into animate St evolve we click into the motion Laura paste all the motion Laura checkpoints into this folder we go back to comy UI reopen comy UI close the terminal to restart comy UI the right way run the B8 file once again to restart Cy UI we have everything installed now we are still going to test by hitting the Q promps to see if this is all going to run through all right so so we come across another challenge here we can see the highlights on the Noe here which tells us where the problem what we are experiencing is to choose a motion Laura we can do a zoom in here maybe an anticlockwise camera movement also we are going to choose 14 which is my favorite as always let's see if we are still going to have any problem going to hit Q prompt and now we are inside a Cas sampler which means so far everything is good and we just have to wait for the output all right so this is our default version and this tells us everything is now set in place we are going to come to our checkpoint and we are still going to use trim like go into our motion lus now we already have everything set up here and coming down here we're going to change our size to 512 by 768 we keep this at 32 frames we are going to use this prompt here inspired by our G ta Grand Turismo and I'm going to provide this in the description and these are few versions developing The Prompt of niik as the character in GTA so I'm going to copy the positive prompt uh from my notepad paste this prompt into our positive and also I'm going to P this here control V keep clip skip at default which is -2 a loader we also keep this at default we come to the cas sampler I'm going to push this to 30 in our video combin I'll keep this at eight frames prefix name we can change this to GTA we just hit Q prompt to see our results from our prompt all right so this is quite interesting what if we wanted to add a Laura this is the Laura I want to use once you have this downloaded put this into your comy UI directory another way is to direct confy UI to locate all your files from automatic 1111 to create a node we can do this in two ways we can either um double click and we search the name or we can also right click and we can add a node here double click here I'm going to search for Laura Laura loader select the checkpoint we want to use our GTA 5 artwork and I'm going to reconnect this here to the model as well usually the color is just a paint for me to follow up with where output and the input is supposed to be I'm also going to connect this back to the yellow connection we have our Laura connected Q pumped again to see what this is going to give us we can see from here our results from adding the Laura this is looking quite impressive I'm very happy with this and I'm going to keep this to change the seed from f fix to random if you are not happy with your generation and what if we want to upscale this to a higher resolution and come to this page we are going to find all the upscale models uh we can use in comy UI click any of the UPS scalers you want download this here paste this in our comy UI directory if you don't want to put this here open your yaml file redirect your UPS scalers uh from your automatic 1111 so back in comy UI we can start to build the upscale nodes double click I'm going to search for upscale I'm going to choose upscale latent by our second node a k sampler double click I set for K sampler our third node I search for vae I'm going to look for the decode our next note is to put all of this into our app scale model which we download double click again we look for an app scale and from here we set the upscale model loader it's going to allow us to select any of the up scale models we just downloaded we can just also drag this which is going to tell us uh which node is best it connects itself going to go for the output we don't need a save image cuz once we select a save image it's just going to produce frames for us to recreate this node double click and we are just going to search for the VHS combine if you don't see the VHS comine come into your UI manager install custom nodes I search for this for make sure you have this installed close this you double click you search for VHS and you are going to find this option here after you install you restart C UI for it to show up and this is going to convert our frames into a video I'm going to change the prefix here to upscale you can always go for any settings you want to choose here we can't connect the vae to the load up scale model there's no input here always make the colors guide us to what we need to connect to each other so the decode goes into the same image I'm just reversing connecting the noes which sometimes I find more simple we already have the vae here this is going all the way to the vae decode here connect this all the way to this connect the colors once again and the positive goes to the positive prompt uh the negative goes to the negative prompt here and the latent goes to the latent at a low resolution of 512 by 768 we want to take this in here to this and from here it's going to upscale this by 1.5 our model is coming from animate def loader and we can connect this twice to this case samp this upscale process is just by preference um make sure you also have your K sampler matching up to the K sampler here so if you have 30 steps make sure your steps is also 30 here also with the Deno we take it down to 0.8 because we don't want the app scale to change drastically from what we have I'm going to hit QBE pump this might take a little bit of time once you have your preview and your happy you can always move on to set up your upscale nodes so I'll just fast forward this for us to see the outcome all right so once this is done this moves to the case sampler it's processing here as we can see to give us the final results which would be the high resolution so this will be our final output uh both from animate diff and also using the upscaler and I'm quite happy with what we have and I'm going to keep this we can either right click to save preview we can save this to any of our folders GTA this is way huge we can see this has been upscaled all the way to 3072 as always don't hold back you can always explore with other models for different outputs you guys can let me know your thoughts in the comments smash the thumbs up if this was valuable you may subscribe if this is your first time as well for more tutorials and you can find out another way of animating in comy UI from the video on the screen now

Channel: goshnii AI

Views: 1,639

Keywords: comfyui, animatediff, stable diffusion, generative ai art, ai art, oliviosarikas, olivio tutorials, stable video diffusion, stable diffusion tutorial, ai, ai animation, ai video, ai video generator, artificial intelligence, text to video, animatediff comfyui, comfyui tutorial, comfy ui, hugging face, prompt engineering, ai image generator, ai generated video, image to video, ai tools, stable diffusion video, stable diffusion video tutorial, video generation, grand theft auto

Id: mMsCB8RNjpI

Length: 11min 15sec (675 seconds)

Published: Sat Dec 09 2023