Setting Up Stable Video Diffusion + ComfyUI Beginners (TUTORIAL)

[Music] by the end of this video we should have a good understanding of using stable video diffusion in comfy UI also how we can generate some of these amazing AI videos for our own projects first of all I will visit the GitHub page for comy UI make sure to already have Python and git installed and if you're already coming from automatic 1111 I guess you already have that installed once we are down here for installing windows by clicking the link here this will help us download a z file for comy UI direct this download to wherever you want to save your file right click this to open with my VRA create a folder wherever you want to have your comy UI directory I'm going to open this drag this into this folder to start the installation process once this is completed we can go into this folder see we have two versions of the Run B8 files CPU makes you run confy UI on your CPU which might be slow run Nvidia allows you to run confy UI on the Nvidia graphics card we come down to savit Ai and from here we can download any model to use in comy UI once you here you can just click on the download save this two comy UI same folder you locate your comy UI and you come down here to your models and save this to your checkpoint another option is to redirect UI so it can locate all the models we already have downloaded from automatic 1111 to do that we are just going to click on this file here which is the yl. example duplicate this file crl C and contrl V backspace all of this rename this file to read as yaml without the example I'm going to double click this file to open with the notepad here we are going to direct our path from automatic 1111 I'm going to start with with a checkpoint which when I come here it should be in the models stable diffusion highlight on the directory location here I'll contrl C this and now when I come down here and I'm going to contrl V I'll also do this for my vae I'll go back to models if I go into my VA downloads copy the path here crl C and I'm going to contrl V also for the lowest we are going to do the same thing highlight this to change the path my lower path I have a few ups scalers here which I'm also going to contrl c contr v so make sure to save this we are ready to open comy UI for the first time and I'm going to use my Nvidia graphics card we'll be patient for this to load once this is done loading this will open in a web browser we can load a default workflow by clicking this tab okay if we drag all the way to the left our first note node is the checkpoint node we have all the checkpoints here because we change our path in the yl file to recognize the checkpoints we already have in automatic 11 if we scroll out and we come down here uh node two will be our positive prompt and also node three will be our negative point you can also right click on any node you choose title and here you can name this to anything you want we can also rename this by right clicking title this to negative right click on any of the notes if we come to color maybe you want your positive to be green and also you want your negative to be red and if we come to the K sampler this is where everything happens before we have our final generation if we scroll down here we come to the empty latent image this allows us to choose the size of our image by default this is one by one up to the vae vae gives a punch to your promps uh through the saturation and also the Vibrance to any check checkpoint selected if we come down here this will be our last note this is where we see our final output by default we already have a promt here to generate our first image I'm going to change this to dream shaper just hit Q prompt we can see the process has already started by seeing this green highlights over this node and we just wait for this to go through all the nodes until we get to save image we can see our final output here and if we should hit Q prompt again this is going to reload the case sampler to give us another version of the same prompt and when we click on view history it's basically going to show us all the process we've gone through so we can click on load to go back to any action in time and this is a very very useful feature in comy UI I have a prompt here which I want to use I'm going to highlight on this control V actually if I zoom in uh this is the prompt I want to use for now and also I'm going to change my load checkpoint down here to excel model we have here let's also change this to a different size 1024 and make sure to click okay 576 and once we are done here I'm just going to push the steps to 40 as well I just click and drag to the right or this Q prompt usually it takes a little bit of time with the generation so just be patient once you hit Q prompt all right so this is what we have from our test prompt uh to also find this in your directory from confy UI we go back to comy UI we come down into the folder we go to the output and down here we are going to find um everything we have exported so far another thing we can do is to drop this image into comy UI and we are going to have the same work flow that we have here so to demonstrate I'm just going to clear this and it's going to ask me I'll say Okay I come back to the output I'll just drag and drop this in here we have the same replica of the workflow we used for that particular image once we are doing this for the second time you guys can see this is way faster and let's see what we get from the second attempt by dropping in the picture into kyui we have something similar in the same direction so this is the similar process we can also use for stable video diffusion to make an image into a video what we need first is to have a workflow and also we need stable video diffusion models which is going to recognize an image to make it into a video if we come to this page here we can have our image into a video format either in 14 frames videos or in 25 frame videos before that we need the stable video diffusion models and we are going to get that from going to hugging phase when we are here in hugging phase uh we can see the description of the models here and when we go down here into files and versions We are going to download this file which is needed you guys need to be careful because this is for 25 frames which is XT and this is for 14 frames which is SVD so these are two separate files download each of these models separately and save them into the directory of your checkpoint comy uip pad we come here to models checkpoints we should save them here and secondly we need to get a workflow you can find different free workflows online and I will be using a workflow provided by enigmatic which the link will be provided in the description a big shout outs to enigmatic for helping the community with all his knowledge as well you can create a folder to keep every workflow you find online I'm just going to open if you come across this challenge seeing red notes what we are missing is to install the comfy UI manager it's on S AI available we can download this to any directory we want to save this to right click and open this with winra open your comy UI directory where you save everything just drag this file in here just double click this to run the installation of confy UI manager to check if this is accurately installed come to the confy UI folder here we go into our custom nodes have the folder here for comy UI manager and now we can close all this we come back to comy UI and close comy UI I'm also going to close the terminal quickly and I'm going to go back into my folder run my inedia GPU V8 file again we see our red notes but down here now we can see we have the manager UI option down here select the manager tab click on the install missing custom notes these are what we are missing from the workflow which was provided to click install second one we also click install and our last one will be this here which we click install all right once all of this is done please restart comy UI and refresh the browser and before we restart I'm also going to update comy UI we have installed a missing noes we have updated comy UI close the browser restart comy UI once again we have no more red boxes which means we have everything installed correctly here we are ready to put in our image we have our checkpoint loader image to video model we go for XT which is 25 frames we are going to choose the picture we want to use any of the models here everything has already been fixed together with the nodes we don't even need to do anything anymore to make sure you have your 25 frames here you're using the same file size as is recommended by St video diffusion for now this is the best size to work with we keep everything at default we just click on Q prompts to see what this workflow is going to give us all right so this is what we get from our generation this is an image to video um workflow but what if we want to put in a text to see the image and then uh you know get into a video generation I'm going to clear this workflow here from what we have and what we need to do is to find that particular workflow that's going to allow us if we come down here we can see um this is a workflow that's available for us to save you right click and we sa this create a folder named this workflow also we have this for 24 frames which I'm also going to sa here I'm going to name this as 24 if we come back to confy UI I'm going to click on load I will select the one for 24 frames I click open this is quite different from the we worklow we had earlier on which was just image to a video but from this workflow if I zoom in you can see we have a text to image and it allows us to put in our prompt and we have a preview image and from the preview image it takes us down to a video image which then we see the video here so I'm still going to use the Excel model we used and zooming in here I'm going to select the sdv XT which you guys can see is already selected next thing we are going to do is to put in our PR we leave everything else at default we also have our size here to be the same steps we have 15 let's see the results we are going to have from this compared to the workflow we had initially you hit Q prompt so this is the results we get as well using this workflow the only thing is to put in our prompt in this workflow and also with the previous workflow we only have to put in the image to find this image we go down to confy UI go to the outputs in the outputs you guys can see we have all our Generations here so some generations might work better than others I would suggest to keep experimenting and there are some amazing Works already out there using stable video diffusion if you found all of this valuable don't forget to give this video a like as well subscribe for more future videos I'll see you guys in the next one

Channel: goshnii AI

Views: 2,418

Keywords: comfyui, animatediff, stable diffusion, generative ai art, ai art, oliviosarikas, olivio tutorials, stable video diffusion, stable diffusion tutorial, ai, ai animation, ai video, ai video generator, artificial intelligence, text to video, animatediff comfyui, comfyui tutorial, comfy ui, hugging face, prompt engineering, ai image generator, ai generated video, image to video, ai tools, stable diffusion video, stable diffusion video tutorial, video generation

Id: Nvm7dmwEX2Y

Length: 12min 38sec (758 seconds)

Published: Sun Dec 03 2023