STABLE VIDEO DIFFUSION | COMFYUI

enigmatic e hey everybody today we're going to be talking about stable video diffusion and if you don't know what this does it basically makes a video out of one image as you're seeing right here I have this one image that I generated normally like I do through comfy UI and by the end of it I get this pretty cool right if you're not familiar with comfy UI I do have a tutorial there should be a link above that shows you how to install it and how to get started so I'm going to tell you what you need to download the workflow is in the description I want to give a shout out to keyi he helped me with setting everything up and and also he's always helping people in the AI community so I want to definitely recognize this person and I apologize if I mispronounced the name so one of the things you want to do is you want to come to this page I'll put a link in the description and you're going to come here and then you have two models here you have this one that says the one tuned to generate 14 frames and the one for 25 frames uh I would download both of them so that you can test them out uh you just come here go to download and you want to download these wherever you have your your checkpoints and if you like you can even download these images and already have an automatic workflow but before you load up the workflow you have to make sure you come to manager and you update comfy UI once you update everything you should have the new stable Video diffusions available just make sure that after updating you restart your comfy UI once you've done that and you start everything now you can bring in the workflow you can load it up through here or you can simply drag an image or a Json file into the browser like this and then you're going to have your setup you're going to have this setup right here all right so once you come here the first thing you want to do you want to put an image from uh something that you've already done for example I'm going to do this this one of this Dragon for the image to video model I'm going to be using SVD XT and all the settings should be already set for you right here so let's just run it as is and see what we get as you may notice there is no prompt here uh so you don't have control really of what's going to happen the only parameters you can play around with is your motion bucket there's the augmentation level and the denoising also affects how the final output's going to look like so as you can see right here this is like a moving picture but let's say you want something more to happen in this image in that case you can try to play around with some parameters uh I'm not sure what exactly what's going to work but let's try like setting these this motion bucket to 200 and let's see what happens as you see here there's also the amount of frames that are being generated in this case it's 25 video frames as you can see there is more movement happening but I want something even more than that putting a random seed might be a good thing and as you can see changing the seed can drastically change the results and as you can see now you're seeing more of this fire and the smoke moving let's play a little bit with the augmentation level let's bring it to 010 and see what happens all right look at this I'm not sure if this happened because of the of the seat change or if it was because the parameters I changed so I'm just going to go ahead and lock this so now I have that seed fixed so I did bring up the hotation to 0.40 because I just wasn't seeing that much movement now you're starting to see the wings move slightly some images work better than others some of them you have to kind of play around with the settings a lot some of them just right out the gate just give you exactly what you want let me try to find something that I think is going to I know it's just guaranteed it's going to work this is the kind of stuff where it does pretty well without you missing too much with the parameters so I'm going to bring this down again motion I'm going to put uh 400 and you see like I predicted this is doing way way better and I'm getting results that just look way nicer it looks like more alive and it looks like something's actually moving like it's supposed to like again these are the parameters you want to play with you want to play with augmentation uh if you can keep augmentation low and still get good results I would try to keep that low pay attention to the motion bucket and and if you have to just play around with other parameters like the noising like I noticed that the noising also affects things slightly so just understand that this is something fairly new that new things are probably going to be introduced to this workflow there's going to be new nodes that can be applied to this to just give us more control I'm pretty sure this has so much potential I'm just so excited what the future is going to bring with stable video diffusion because we're starting to create some amazing visuals with this we also have our SVD model as well which is supposed to be 14 frames so let's put 14 here and let's see what this gives us and as you can see you get very cool results out of just one image yeah that's so cool the future is here man yeah exciting stuff so there's still a lot of things that are being discovered there's a lot of things I still don't understand about this but at least wanted to give you guys something to start off with so that you guys can have your hands on this and play around with it yourself so yeah that's it for today thank you for watching like always take care God bless and [Music] peace

Channel: enigmatic_e

Views: 26,463

Keywords: comfyui, animatediff, stable diffusion, generative ai art, ai art

Id: imyQuIiuRnA

Length: 5min 22sec (322 seconds)

Published: Sat Nov 25 2023