Stable Diffusion AnimateDiff For SDXL Released Beta! Here Is What You Need (Tutorial Guide)

all righty then we've all seen those YouTube videos about anime diff using the stable diffusion 1.5 models right guess what the anime diff custom node in comi just got an awesome new update and now it supports the sdxl model how cool is that so we had the Hot Shot XL FPS 16 version model before and we talked about it in our previous video on automatic 1111 but honestly that 16fps model didn't quite hit the mark but fear not there's a brand spanking new animation model from animate diff hugging face for sdxl that's currently in beta and you know what I just couldn't resist giving it a go in its early stage so let's dive in and see how this new animate diff sdxl Beta And The Hot Shot XL fair with the workflow that I built in previous video shall we all righty then let's get this show on the road we fired up comfy UI as we always do hit that bat file and boom the command prompt window comes to life and voila look at that comfy UI web interface right in front of us now the first order of business is to download the animate diff sdxl beta model and guess what this is my first time trying out this beta version 2 I haven't seen any other YouTube videos delving into it yet so we are exploring together today no worries at all the download process is a piece of cake my friends I'll make sure to provide the hugging face link in the video description so you can download it while enjoying this awesome video only if you have a high bandwidth though all right let's download the sdxl V10 beta. CPT model and hey if you don't have the OG stable diffusion 1.5 model yet grab the sdv5 V2 while you're at it save those model files into the animate diff folder inside the comy UI custom nodes and make sure to put them in the models subfolder as you can see I already have the trusty SD 1.5 V2 model right here in my setup once your download is complete head on over to comi UI hit that refresh button like a pro and voila the sdxl V10 beta right there in front of you congratulations that's your first steps now we are going to check out other nod setting now go to checkpoint loader select sdx L and vae model also animate diff sdxl in beta schedule in control net model we will need to download sdxl line art model as well here I downloaded the diffusers EXL line art model remember put your control net model file in here come back to comy UI refresh you will see the new control net model in the drop-down menu as you can see my testing on the screen the line art preview image works let's clarify the process once again now for the checkpoint model make sure it's the sdxl model specifically as the beta schedule is geared towards line art animate diff sdxl in this case since your checkpoint model includes vae you'll need to reconnect the node using the vae checkpoint model as you can see we have options such as line art Hot Shot XEL sdxl and SD 1.5 in the beta schedule within the animate diff group don't forget to select animate diff sdxl in the beta schedule as well for the model name we'll be using the sdxl V10 beta. CPT as for the other new options I haven't had a chance to explore them yet right now the main focus is to ensure this current workflow functions smoothly with the new sdxl model yep it's working as you can see in the command prompt Windows the sdxl ckpt model file loaded and here the case sampler loading now let's wait for the outcome result so here's the first test with 10 image frames okay looks cool space background as what I expected and we keep the same range in 0.5 to 0.55 in control net strength all right let's move on to testing the hot shot XEL model using the comfyi animate diff custom node we've covered the Hot Shot XL model before and you can find the animation model file on the hugging face page for downloading to switch between these models you'll need to configure the checkpoint loader and animate diff loader in conf UI accordingly I've tried enabling the apply V2 models option in the animate diff loader and it does perform better in my experience and if I recall correctly this is a new Option introduced in the latest update of the animate diff custom node exciting stuff once the generation process begins and you take a peek at the command prompt window you'll notice the Hot Shot XL model loading into the process now let's patiently wait for the outcome and see what we get and there you have it folks an animated video created using the Hot Shot XL model um um truth be told it's not that impressive if you ask me to choose a winner between the two I definitely go with the animate diff sdxl beta model I haven't changed the text prompt in both the XEL model tests and all the settings in control net and the checkpoint models remain the same um while there isn't much detail the body movements are still flicker free I believe this is what the anime diff with line art control net excels at and it's not necessarily attributed to the Hot Shot model so in terms of credit I don't think Hot Shot deserves much recognition for that aspect and now let's talk about the good old SD 1.5 model with animated diff setting oh boy the pink lady dance animation using this model is just incredibly smooth all the motions and everything it's like butter I'm really excited for the upcoming update of the anime diff sdxl model after this current beta version hopefully they can improve improve the image frame resolution and optimize it even further for realism checkpoint models so far I truly believe that anime diff is a game Cher in AI animation it has opened up a whole new world of possibilities one of the greatest things about this custom node is that we can create animation videos that are longer than just six 7 or 8 seconds isn't that awesome of course to handle greater processing you'll need a high-end graphics card if you're interested in trying out this workflow and creating flicker-free AI animations be sure to check it out in our patreon community until then my friends I'll catch you in the next video Happy animating and peace out

Channel: AI How To With Benji

Views: 12,304

Keywords:

Id: K-pUyk_sK6k

Length: 8min 32sec (512 seconds)

Published: Mon Nov 20 2023