Bring Images to LIFE with Stable Video Diffusion | A.I Video Tutorial

stability AI has finally released their own video model that lets you bring images to life and create videos from text prompts and I'm going to show you how to do it at the time of this recording there are essentially two ways to run stable video diffusion the first option is totally free but it does require some technical knoow and computational resources to go with this first method you will need to install two main things on your computer confy UI and confi manager I will leave the links to those two below and I've already covered the installation process step by step in an older video so make sure you check that out for more guidance once you're done with the installation head over to this hugging face page to download this table video diffusion image to video model find the SVD XD file and right click here choose save link as make sure you place the file in the correct path by going to your comy UI folder open models checkpoints and and hit save after that go back to your comy UI folder and run this file to launch the UI on your browser if you installed comfyi before watching this video open the manager and click on update all and restart comfyi to apply all the updates now if you want a much easier way to get stable video diffusion up and running with way fewer clicks you can try out this cloud-based solution called think diffusion here you have everything you need including pre-installed models and extensions that are ready to use with think diffusion you can run different environments like automatic 1111 and comy UI the best part is that you get access to high-end gpus and memory resources so you can run stable diffusion from almost any device now think diffusion is the sponsor of this video and they've been a great supporter of my channel for quite some time I even made another video where I put think diffusion to the test to see if it's really worth the investment so if you're still unsure about whether it's the right choice for you I highly recommend watching that video so I'm going to use think diffusion for this tutorial but if you want to run locally you can still follow along because the process is going to be the same for both methods so here you're going to choose the confi option the available machines offer different resources so you can choose what works best for you I want to maximize the speed so I'm going to go ahead and launch a turbo machine here you can set a limit to your session to help man your time I'm confident that 1 hour is enough but you can change this later if you need to let's go ahead and click on launch it usually takes around a minute after which you will be able to see comy UI on your browser the file structure is pretty much the same as the local version and you can close the file browser for better viewing to get started with image to video we need to replace this default workflow with a different one this page right here has the basic workflows that will help you get started and it's going to be very easy head down over here where it says workflow in Json format right click save link as and you can actually place it anywhere on your computer then simply drag and drop the Json file into think diffusion to load the workflow now we can already use this workflow to animate an image but I've made some changes to the settings and created my own version of the workflow that you can download from the description box once you have it simply drop the Json file here to load the workflow if you're doing this locally you may encounter errors about missing nodes to fix that open the manager click on install missing custom nodes and it will provide a list of all the missing nodes that your workflow requires click on install and make sure to restart compi to apply the changes in most cases you won't have to worry about this if you use think diffusion as most popular nodes are pre-installed now let's take a look at how to use the workflow this node is used to load the stable video diff Fusion model down here you can select the image you want to animate I have a few options to choose from and I think we can go with this pirate ship I've generated these images using mid Journey you can do the same or bring in your own images stable video diffusion works well with portraits as well and by the way if you want to know my secret to writing really good prompts check out this video currently the video model works best with 16x9 images so make sure you specify that ratio when generating images with mid Journey for the resolution you can just leave it as the default setting now the two main settings that you can play around with are motion bucket ID and augmentation level the motion bucket ID controls the amount of motion in the video and I find that 150 is a good starting point as for the augmentation level the higher it is the less the video will resemble the original image so if you want to see even more motion you can increase this value and I usually set it somewhere between 0 and 0 .1 another change I made was setting the steps to 25 which should result in better overall quality I also set the CFG to3 but feel free to experiment with this as well unlike the original workflow I'm using a video combined node here so you can export the video in MP4 and of course you have other format options to choose from and that's pretty much it for the settings now you can go ahead and click on Q prompt to start the process and execute the nodes one by one and this is how the image looks like in motion I'm really impressed by the end result and the capabilities of this model especially how it brings the clouds and waves to life it's simply amazing just a heads up at the time of this recording the videos you get are limited to 25 frames however I'm pretty sure there will be other models and workflows in the future that will allow you to render longer videos to access the video file simply open the comy UI folder go to output and you can download your videos directly from here now let me show you what happens when we crank up the motion bucket ID to 200 as you can see we get even more Motion in the video I also played around with increasing the augmentation level and it seems like it adds more camera movement this may vary depending on your input so don't hesitate to experiment with different values now because the quality of these video outputs isn't the greatest you can use a AI upscaler like topas video AI to enhance the video and increase its resolution let's double the video dimensions and increase the frame rate to 25 for a smoother playback when it comes to the AI model I usually go with Thea and bump up these three settings to 10 you can also change the file format if you prefer and then click on export top pass video AI works really quickly and when you compare the original video to the upscaled version you can really see the difference difference as I mentioned earlier in this video you can also use stable video diffusion to create videos from text prompt I'll provide a link to the workflow I'm using so you can give it a try these settings might need to be adjusted slightly but the main things to experiment with are the bucket ID and augmentation level when you run the workflow it uses the base sdxl model and text prompts to generate an image first which is then sent to the video workflow to bring it to life most of the time you will get really good results especially considering that this is a newly released model keep in mind that the generated image may change each time but if you want to keep using the same image for your videos you can simply set the seed here to fixed after you're done with think diffusion make sure to stop the machine as you can see I still have 35 minutes left in my session which means I'll be charged less than a dollar if you're interested in exploring other tools for generating AI videos check out my anime diff tutorial and if you find it useful don't forget to give it a like stay creative and I'll see you in the next video [Music] peace

Channel: MDMZ

Views: 41,765

Keywords: stable diffusion, ai animation, image to video, ai video generator, stable diffusion video, turn image to video, stable video diffusion, stable video tutorial, ai video tutorial, ai video tools, stable diffusion video generation, ai video stable diffusion, stable diffusion comfyui, stable diffusion comfyui install, ai art generator free, free ai video generator

Id: XPRXhnrmzzs

Length: 8min 15sec (495 seconds)

Published: Thu Dec 14 2023