Style Your Videos with Stable Diffusion & Deforum (Video 2 Video)

get ready to transform your videos into cinematic wonders using stable diffusion and deorum in this tutorial we'll guide you through the secrets of video styling revealing two methods to add flare to your [Music] content today we're uncovering the magic behind transforming your videos with stable diffusion and deorum Brace yourselves for two game-changing methods that will elevate your content to new heights to Kickstart your video transformation Journey make sure you have stable diffusion deorum and control net installed you can find tutorials for that in the description below for control net we will try out three different models for the first method we will only use the tile model this is the faster way to style your videos but it's less consistent for the second method we will need the soft Edge and temporal net models we will use these two models simultaneously in control net it's a bit slower but it's very consistent as you will see later in the video download these models from the hugging face website and put them in your control net models folder before we move on to deorum we first need to do some quick testing in the imageo image tab you'll need an image or frame from the video you want to transform this can be captured using software like Premier Pro open your video in any software of your choice and Export a frame alternatively you can use the built-in Snipping tool on Windows for quick screenshots place the image in the image to image Tab and adjust the following settings the most important setting is the checkpoint you want to use this defines the style that your video will be rendered in I will be using the tune you checkpoint but you can browse for any checkpoint you like on civit AI next up enter your prompt and negative prompt I've also included the LCM Laura at a value of 0.7 this is optional but I highly recommend it because it makes the generation go way faster to learn more about the LCM Laura I'll link a video in the corner right about now set the sampler to d a use eight sampling steps for width and height match these to your input image Dimensions or opt for resizing by next adjust the CFG scale within the range of 1 to two and set the denoising strength of one if you are not using the LCM Laura increase the sampling steps to at least 15 or more and the CFG scale to about seven now it's time to use control net scroll down and open up your control net Tab and enable it first off set the control type to tile / blur and the pre-processor To None make sure the right model is selected the sd15 tile if you don't see it click the refresh icon here and make sure the file is located in the control net models folder next up increase the control weight to 1.5 to 1.8 and select control net is more important by fine-tuning these image to image settings you lay the groundwork for optimal deform results now let's generate and see the result that already looks really good for just using one control net unit now let's quickly explore a more consistent way before we head over to deorum you can leave all the settings the same except for control net we will be using two control net units make sure to enable them both for the first unit we want to use some of the default options so set the control type back to all the pre-processor to none and for the model select the temporal net model slightly decrease the control weight to about 0 .7 and set the control mode to balanced for the second control net unit use the following settings set the control type to soft Edge change the pre-processor to soft Edge head and make sure the soft Edge model is selected for the control weight I suggest a value between 1.5 and 1.8 lastly set the control mode to control net is more important now let's generate and see the example in this case I like the first method better this may vary per video as I will show in later examples this is why testing at first is a crucial step in this process if you're satisfied with the results we can move over to the deorum tab to make your life a little easier I've made two settings files that you can download for free I'll leave the links in the description simply download the files and put it in your stable diffusion web UI folder copy the path of the file and paste it in the deorum settings file section and click on load all settings after successfully loading all the settings there still are a few settings you need to change yourself let's run those down real quick in the Run tab you may need to change the width and height to match your video's aspect ratio here you also have the option to rename your output video under batch name in the key frames tab you have the option to change the Cadence if your video has a lot of movement I suggest keeping this at one you can also play around with the strength schedule this is how much you want the previous frame to influence Ence the next frame for slower videos I recommend increasing this a little bit as it can improve consistency by a lot I'll show you an example of this later you can ignore the max frames as this does not apply to this workflow for even more consistency you can select video input for the color coherence in the coherence subtab also quick note if you have an AMD or Intel GPU you want to disable used depth warping in the depth warping and fov subtab in the prompts tab you need to change the prompts to match the one you used in testing make sure to include the Laura and use the right formatting for the init tab we need to locate the video init subtab and paste our video path here rightclick your video and press copy as path on Windows 10 use shift and rightclick then paste the path and make sure to remove the double quotes set the control net settings to match the ones you used in testing we don't need to input our video path here as it will copy over the same path used in the init tab in the output tab make sure that the FPS matches with the video you're using to find the frames per second of your original video navigate to its properties select details and locate the FPS information displayed next to frame rate Additionally you can select to keep the same audio as your original video under add soundtrack now you're ready to generate that took less than 15 minutes pretty quick for a 7-second video now click here to see the result don't forget to like And subscribe if you found this video helpful so far that looks pretty good already I've interpolated this video to 50 FPS let's see the difference to really show the difference between the two methods I have run this video through deorum again with temporal net and soft Edge enabled the outcome was not as nice as the tile only video but don't worry I've got another example to show where the second method is obviously Superior there's still a tiny bit of glitching that's due to the fast movement causing motion blur in the the original video but overall it's really consistent just look at the fluid motion now let's watch a final comparison to help you decide what method you should choose and if you want to try this out for yourself but don't have a good enough graphics card check out think diffusion a handy website that lets you run stable diffusion in the cloud sign up with the link in the description for a free trial and on top of that you get a 20% bonus on your first deposit that's a wrap for this video I hope you enjoyed it and learned something new if you have any questions feel free to leave them below peace out

Channel: Next Diffusion

Views: 4,388

Keywords: stable diffusion, ai, midjourney, ai image generation, ai video generation, stable diffusion tutorial, deforum, video to video, video style transfer, video2video, deforum video, temporalnet, softedge

Id: aMnzxa0hJDY

Length: 8min 3sec (483 seconds)

Published: Wed Dec 20 2023