How to generate Video with Stable Diffusion! For FREE and in one click | Stable VIDEO Diffusion

stability AI just released new generative model  for generating video and that is absolutely   stunning in this video we will discuss what is  special in that particular model and also how to   test it out using Google collab no so let's go at  this moment the SCE code of stable video diffusion   available on GitHub repository and waights also  available on a hugen face page at this moment   it's only research preview and even stability AI  didn't release any service which can utilize this   specific model first of all according to press  release this video model can be easily adapted   to various Downstream tasks including multiv  view symmetric from a single image with fire   tuning or multi view data sets they are planning a  variety of models that built on a extend dis base   and similar to the ecosystem that has been built  around the table and here you can see the samples   of multi view generation from fune video model  and that looks stunning there are also information   about performance comparing with d competitors p  collaps and gway and according to this test stable   video defusion on power with runway in a case  with 14 frames per seconds and much more better   than pabs in a test with 75 frames stable video  diffusion outperforms all competitors Runway and   pabs and that looks very very promising but first  of all what is special in that particular model   we already seen AI generated video with st F but  there are a lot of difference first of all most of   this video was based on other videos some models  like a cont net extract additional information and   transfer this information through stab diffusion  will generate a sequence of similar images but at   the same time quite accidential and by summarizing  these images we are getting some kind of video   with such flickering effect we are also seeing  anime diff extension which works completely   different in this case anime do just generate a  sequence of images which is around the same seat   with the same prompt and then just combine this  images creating Smooth video but there was no any   logic in this video because all that movements are  absolutely accidental models doesn't understand   that is a arm and that is a body and it might be  transformed body to arm and so on in this case   stable video diffusion understand the context of  the image for example if you see a car most likely   this car should move and as you can see here  this car is moving the same situation with a   train and with this guy and so on first of all a  very important moment you should understand what   this model might animate on your image because  sometimes it's extremely hard to anticipate   something animated for example in this case what  might be animated here I don't know just PX effect   and exactly this stable video diffusion is doing  making just PX in the case of dog quite easy to   predict that dog might move and exactly this  happening in this image the same situation with   a car the same situation with a people you can see  it's a very good animation because M understands   that is a people and understand what might happen  with this people in this moment for example just   move and turn around in this case with a crocodile  there is no any animation because the image   quality is not so good and the model The Source  model is also quite hard true ande something good   but if I take a picture in a good quality are able  to get better animation like this one so in order   to get good results you need to get the model  the image which might be quite easy to animate   and this animation should be as much as possible  easy to predict you can run stable video diffusion   in confi using this specific workflow but in this  video we will discuss another and to my mind much   more convenient way especially for testing in  Google color to run stable video diffusion open   the link below available under this video this  is a colop notebook which have been made not   by me but by this guy from stability uh MK Shing I  don't know how to spell it properly but very thank   you for that so to use this notebook you need to  first of all run setup then we need to run soall   collab hack for SVD then we have to choose the r  for our model there are two options SD and SD XT   first one uh SVD is for 14 frame generation and  SVD XT for 25 frames generation even in a free   Google Cloud Tire you are able to generate video  with 25 frames so let's choose this one then the   next step is load model as usual press load here  sampling functions run and finally run interface   for stable video diffusion it takes some time to  download all needed components about 10 minutes I   also created notebook with one click installation  and that notebook available on my patreon page   as usual and here you can also find a lot of  different notebooks for St diffusion and different   user interface which will work in a Google color  free Tire in order to avoid possible problems   with video generation I would recommend to choose  specific resolution 1054 by 576 that resolution is   minimize the chance of possible problems you  can do that in any image editing software I   decided to do this in a Cana just create design  with custom size 1054 by 576 create new design   prepare your images for video generation and then  export them in proper resolution I just prepare   several images for testing and let's say them we  needed resolution here we go we just loaded our   interface right here in a Cell but you can open  this link and open gradi your interface in more   convenient way well what we need to do just drop  image here and let's take any image you like for   example this one here we have advanced options  where you can adjust number of frames number   of steps seat and number of frames decoded at  a time you can experiment with that parameter   but be aware that you might have problem with  uh video memory especially if you make bigger   number of frames so then we just need to press  run button here and wait about 7 minutes for 25   5 frames per second here you can see our results  which looks really nice and you can experiment   with different parameters and different images in  some moment you definitely me the problem it will   run out of memory and it's absolutely okay and  unfortunately you nothing can do with that and   the most optimal solution for this problem is just  to restart your session and to be honest it is not   the best solution because in this case you will  have to download all models and requirements once   again and when a lot of time as you know I have a  life hack for you quite simple just create a cell   enter exit commment and with this command you will  restart your session but all downloaded files and   all downloaded all installed requirements will be  there and you will save a lot of time because you   will run your interface much more faster and to  do this just run each sales once again that's it

Channel: marat_ai

Views: 8,634

Keywords: stable video diffusion, stable video diffusion install, stable diffusion video install, stable video diffusion colab notebook, AI video, ai video generator, runway alternative, pika alternative, runway free alternative, pika alternative free, ai video free, image to video, image to video ai, pika labs free, runway gen 2 alternative, stable diffusion video generation, stable diffusion video free, stable diffusion video animation, stable video diffusion free, video diffusion

Id: k8z3Bn_Wkbg

Length: 7min 25sec (445 seconds)

Published: Sun Nov 26 2023