Stable Video Diffusion Quickstart (with ComfyUI on Kaggle)

hello creators today's video is all about making videos but with AI introducing stable video diffusion with comfy UI let's Dive Right In and see how this app works its magic we have a handy kaggle notebook ready to create some amazing videos right from the browser check the description for the link and hit copy and edit to get started if you're new to kaggle ensure your account is verified with a phone for GPU access and internet connectivity consider also enabling file persist to save your work for future sessions now let's jump into the action click the Run button next to the installation code block to set up comfy UI by default it installs the albo base model but feel free to switch it if you have a preference this step makes sure comfy UI and all the necessary nodes for video generation are ready with the installation complete click Run next to the starting with remote Mo block and wait for the line that says the app is running on a local URL then follow the link ending with remote. moo to access the user interface grab the text to video workflow from the samples folder on GitHub and drop it onto comfy UI you can explore the workflow by holding down the left Mouse button to drag the screen area and use the mouse scroller to zoom into the nodes you wish to edit to start generating the video click the Q prompt button stable video diffusion transforms images into videos so it's a two-step process the workflow will create an image and then generate the video the video on the left here is the original one created by the SVD model and the one on the right was modified by a frame interpolation node to have a higher frame count for a smoother effect the whole workflow can take about 20 minutes so in the next attempt we'll start by generating images until we find the perfect video candidate before switching to image to video workflows Let's download some extra flare use the download Laura block to give your images specific Styles like Samaritan and Dark Fantasy ensure the Laura file name ends with DOT safe tensor so that comfy UI can find it I prefer downloading models to temporary storage to avoid running out of space but if you fill up your permanent storage don't worry there's a block in the notebook that lets us delete models we no longer need how do we know what the model is called by running the block which lists models and placing the model name in The Path there's another block that lists all big files to help identify anything you might want to remove now let's grab those new models when downloading either checkpoints or Loris I use a private window to make sure the notebook will be able to access the download link if the download doesn't start the notebook won't be able to grab the file either we're downloading a turbo checkpoint model which requires fewer steps compared to standard sdx Excel models and speeds up generation with the models in place let's create an image for our next video fire up the app import the text to image workflow and play around with parameters to enable a Laura type one into the use Laura field this workflow lets us use up to two luras I've chosen Samaritan and Dark Fantasy the free you node below tends to make the images more vibrant and eye-catchy for the prompt I'll describe an ice princess walking through a small town in Winter and I'll choose a landscape mode resolution because we're using a turbo model I'll make sure the step count is low enough and select auler that works best with this model I'll download the best image for the next step you can use the history list to browse recently generated images or download them from The Notebook to make the video Drop the image to video autoscale workflow to comfy UI and drop the image into the load image node just like with images ancestral Samplers work better on people so I've selected one of those this workflow automatically calculates the video's resolution to keep the shape of the original image if you want to crop the image for example to create a square video try the image to video workflow When selecting a resolution keep in mind that the video diffusion model was trained on 576 X12 24 pixels so it helps to stay close to that using a lower resolution will also prevent issues such as running out of memory if this does happen save your workflow to keep the changes you made and restart the web UI step if you need a shorter video you can switch from SVD Josh XT to the SVD model and lower the frame count to 14 which works best with that model I often repeat the video generation with different seeds for varied outputs if the the animation turned out okay but you didn't get the effect you wanted it's worth it to try a few more times with small tweaks speaking of trying again we have one more example this time with a text logo highlighting this inevitable part of the AI creative process this example uses the cyberpunk AI and haros lores we'll import the workflow by dragging an image previously created with comfi to the workflow area each image has the entire workflow that created it embedded as metadata so if you create an image you like and want to tweak the parameters simply drag the image to comi and it will recreate the whole workflow with the needed parameters I'll run this one but increase the step count to nine I think it turned out pretty well the video workflow will also be embedded in the image that gets saved alongside a video now open the image to video workflow and give it the new image all that's left to do is run the workflow and here's our animated motivation to never stop trying and improving let's make sure we have our work saved so we can share it with friends close to the bottom of the notebook there's a Code block that zips all outputs to make this task easier now there's only one file to download if the file doesn't show up in the file manager try hitting the refresh button or reload the whole page in your browser you can explore the images and animations offline on your PC after youve verified you have all your Creations you can run the delete block to free up the outputs folder which will make new videos and images easier to find and that's a wrap on our quick introduction to AI video creation there are lots of other AI tools available so if you liked this one check out our other videos for more exciting AI content thanks for watching Tech Wizards remember to hit that like button subscribe and turn on the notification Bell so you don't miss our next content creating AI update until next time stay curious and keep pushing the boundaries of what's possible with technology see you in the next video

Channel: Pogs Cafe

Views: 1,072

Keywords: ai, generative ai, stable diffusion, stable video diffusion, ai video, ai shorts, image to video, i2v, comfy, comfyai

Id: 3QaZE6gLuZ0

Length: 7min 7sec (427 seconds)

Published: Sun Dec 17 2023