Videos in Stable Diffusion, AnimateDiff with ControlNet

hello there in this video we're going to create animations with stable diffusion and automatic 11 11 UI with animate div first we want to install it if you not yet have if you do install before you should see it down below like right there says animate div if not let's go ahead inside the extensions we'll click on available and let's go ahead load from just downloaded all of them and in a search we'll just type animate div just like this and you can see right here we have our extensions in my case it already says it's installed if you don't have an install it you just need click to install after this you actually want to restart your um stable diffusion from command prom just close it windows and start again after installations you you should see in installed area it will be available down below I always also recommend for you to go ahead click check for updates be sure every extensions that you have uploaded it's up to dat if they're not up to date like right here you can see it says new commit for example for my animated div is available in this case you want to go ahead and press apply and restart and allowed your server to restart it in some cases even after this restart you maybe want turn down server and restarted some extensions kind of pecky after everything is updated we're ready to start using I'm using the checkpoint delate version 5 if you're interested in this currently I don't think it's available on the CIT AI I will provide link down below for you you can go on a hugging pH and download it from there directly be sure if you like this checkpoint to support the after again I will provide links for all of these resources Down Below in descriptions we also going to use it with our checkpoints animate div and as well for the future animations we're also going to use it control net if you don't have it control net which is most unlikely if you us it stable diffusion before you can also go inside extensions available load from and download it from here as well just same path so same way you just use it look for the control net and unload it where is my control net right here like this one says install it with control net you need for for sure to restart your server because it Le have a lot of libraries also with control net I would Rec recommend you to go after in the settings scroll down in an area where you have it control n options click on control net and inside here you can also enable disable some options mostly what I needed it is used little bit more of available control net like three point so this is will help you as well to have it more this is will require restart server as well with your settings after everything is installed let's go ahead and try so first I'm going types so right there this is what I want to do portrait of Superman and we'll just leave it let's switch this to the cars M we'll put it maybe 25 steps and leave it everything same let's go ahead click generate so we can just preview see how it's look and right here we have very nice portrait that we can go to use now it will be nice to animate this for this we're going to use the animate diff let's expand it and we want to use it model is have it multiple different models links will be down below for you download it you can use it animated motions is very nice as well as mm SD notice they're all V5 so it is using the work with a checkpoints with a 15 1.5 version stabil diffusion so sdxl not yet supported with those motion uh modules you will get it some errors if you start doing this to download that checkpoint for animations you need going to the hugging face path it will be provided down below for you as well and download this file be sure you download this file in specific location it will will be in your stable diffusion folder extensions in extension you need find SD webui animated def folder expand and inside the model subfolder you need place here so in this case it will find just by that extension next we have it also save format GIF and I know some people call GI some people call GIF so we won't go in this water at that point I will just refer you to check Wikipedia how you should pronounce this and we uncheck PNG because I don't want save just only one animation file next we're going to enable our animations number of frames let's go ahead 32 frames we'll have it FPC 8 context batch file it's how many frames we'll create it and kind of working between them be sure they all qualify it's one and in some cases CFG scale will maybe one pop up little bit higher but for now we'll just leave a default of seven and close the loop is enable RP so the last Fram is kind of going forward I think this is look good for now let's go ahead and click generate this will take a little bit longer time because it does need to regenerate 32 frames right now and you can see as a render it's created our sheet of the frames that going to use in our animations on the end is going to assemble and right here we have our image created actually I like how the lighting ad you can see it's a loopable very nice animations the few options you can modify see what result you wanted one with CFG scale of course if we drop very low like 2 one it will be huge different between the frames or or in some case if you want to go higher but then will be less animation so you can play with this as well additional options if we look on a context batch if you reduce this to about maybe eight frames you'll have too much distortions on the background it's won't be consistent with this and if you have a too high it will be more blurry and you almost don't see any difference so you can play with this usually it's divide by how many frames you have for example 30 2 divide by half it will be about 16 frames also frames per second currently it says eight it's actually very good for the our animations as a file if you're going with a um file like MP4 or webm then you want to switch maybe to 12 frames per second or 24 so divide by 24 kind of more useful it's up to you so let's go ahead and try something different because beside just creating from from text to text we also can drop video here directly and create from the video let's go ahead and try this for this I put it image inside and we can actually preview a little bit of this image but notice it is automatically adjust 25 frames and have 375 frames so it's quite bit frames total in this case what I want to do is just maybe to decrease I know the quality won't be up there but maybe to the 10 steps you can always verify if you just click let's go remove Superman poser just Superman then steps we can always click and verify it how it will look and let's wait a second till it's processing and here look quite a bit ugly okay let's go maybe to 20 we generate a little bit more I think the deliberate is like maybe about 35 it's where the best quality but for the test for this because time I don't want to spend too much time on this as example I think this one has actually work very well so we're going this as well if you notice it is kind of wide frame for this we're going to say with 768 so we'll just make a little bit wider okay this kind of enabled but next we want to take from our animations and use it in control net so we'll expand control net in this case and notice right here we have it only image so we can put it only image around or it will actually read from what extract from there so we'll go ahead enable this enable our open pose and right here you can see we have it our open pose preloaded let's go click more on the control net also I want to enable Pixel Perfect because with this case it will read our original image and I think this is actually will work good the if you're not sure about reading original you can always take um your file video file on frames and using as a batch file right here and specify your input directory so in this case what I want to do as example I'm going to use the batch file with a single frames so right here is my frames and if you're looking for this this is how all the single frames from this IM from this video is looking to extract the frames if you don't have it I would recommend for you to go and download it shatter encoder link will provide down below it's very easy to use as well all what you need to do you just take the video file you have it drop on the middle next you want to have choose the functions and our functions is will be jpeg images right here we want create check create image sequence spit FY in a place where you want to do it and start function that is all actually nice things about sh encod it can take it and compress back to video from single images so very simple straightforward to use okay after we're done I think they're all working now we're ready to go ahead and click on generate this may take little bit time so it's good time to go ahead and take a break okay so right here is our animations done and you can see it's apply control net and well it's quite a bit sexy dance for our spider Superman going on here it took a little bit time because we had what about 375 frames I think that's what right there says you can see we have some little bit low consistency we can upscale by CFG scale and other things to create little bit better background is not bad and I don't see so much flickering of course some quality is reduction due to reducing and sampling steps and as well context bdch files so for this one probably should have a 32 but this will take a little bit longer time to render but overall it's kind of funny and I'm surprised it did actually very well for 375 frames because usually uh I found anime diff it's better if it's work in amount like maybe couple seconds of animations um and then you can compile all them together that way is performed the best but for this even the long one is actually performed quite a bit nice way and you can see of course it's a loop at animations and done very well thank you for watching this video hopefully you found it useful interesting and please let me know what you created have a great day bye

Channel: Vladimir Chopine [GeekatPlay]

Views: 1,830

Keywords: Geekatplay Studio, Vladimir Chopine, Digital art, How to, Learning Photoshop, 3d tutorials, Digital compositing, How to do in Photoshop, How to do compositing, How to do in 3d, AI art, MidJourney, Stable Diffusion, Dreambooth, Dall-e, Free resources, Free learning, Digital art for begginers, Free tutorials, artificial intelligence, Digital Photography, Photography

Id: 1dnFGxE0vzI

Length: 13min 2sec (782 seconds)

Published: Fri Dec 29 2023