Transform Videos to ANY STYLE with Stable Diffusion and Deforum (FREE)

ready to unlock the full potential of your videos using stable diffusion and deform in this step-by-step tutorial we'll guide you through transforming your videos into stunning masterpieces let's Dive Right In [Music] before we begin make sure you have deform and control net properly installed if you're new to these tools don't worry we've got comprehensive guides Linked In the description to assist you before we begin we need to adjust a default setting of stable diffusion adjusting the initial noise multiplier this parameter fine-tunes the level of detail in your images however for deforum Less detail works better to prevent flickering by default it starts at 0.5 but we'll change it to zero for our purposes navigate to the modules folder and find the shared.pi file right click on shared.pi and select edit then press Ctrl and F together to search for initial noise Make sure wrap around is selected after finding it modify the minimum value from 0.5 to 0.0 once you've made the change save the file and then close it to easily access the initial noise multiplier in stable diffusion enable it in the Quick Settings list within the user interface go to the settings tab locate user interface scroll down to the Quick Settings list enter initial noise multiplier and press enter apply the settings and refresh the UI let's begin setting up the prompts and settings for your videos You'll need an image or frame from the video you want to transform this can be captured using software like Premiere Pro open your video in any software of your choice and Export a frame this image we will paste in our image to image Tab and adjust the following settings set the sampler to Euler a use 20 steps for width and height match these to your input image Dimensions or opt for resizing by if your input image is large consider reducing the output size a recommended maximum Dimension is 576 by 1024 for a 9x16 image ratio next adjust the CFG scale within the range of three to five and apply a denoising strength of 1. let's dive into the control net settings we'll be working with two control net units for the first unit here's how you can fine tune it ensure Pixel Perfect is set to yes choose the control type as tile set the preprocessor as none opt for the model labeled control tile adjust the control weight within the range of 1.5 to 1.75 lastly prioritize control net by setting the control mode as control net is more important for the second control net unit follow these settings set Pixel Perfect to no choose control type as open pose for preprocessor select open pose opt for the model labeled control open pose leave the control weight at 1 and prioritize control net by setting control mode as control net is more important make sure both control net units are enabled and hit generate to truly capture your desired style let's talk about Laura if you're not familiar check out our in-depth guide on how to install and use Lora models in stable diffusion I'll leave a link in the description I will show off this one piece style Laura I found on civit AI now let's tackle the deforum settings it might seem overwhelming but don't fret we'll walk you through each tab step by step if you don't want to do all of this manually you can load in a settings file that I have created for you you can find a link in the description first off let's tackle the Run tab here's what you need to do use the sampler labeled Euler a set the steps to 20. adjust the width and height to match your video input in my case 576 by 1024 keep the seed as -1 when you name your batch make it descriptive so you can easily locate it within the img2 IMG folder now for the keyframes tab let's make some adjustments set the animation mode to 3D opt for Border mode as wrap depending on your video's movement choose cadences two or one for videos with lots of movement one might be a better choice within the keyframes tab there are additional sub tabs where we need to tweak some settings in the strength sub tab adjust the strength schedule from 0.65 to 0. in the CFG sub tab change the CFG scale schedule from 7 to around 3 to 5. in the seed sub tab set the seed Behavior to fixed to ensure consistency in the video now let's move on to the tabs below in the motion sub tab set translation Z to zero for the noise sub tab aim to keep the noise schedule between 0 and 0.03 generally zero tends to work best in the coherent sub tab change the color coherence To None however if you prefer to maintain the original colors from the video you can opt for video input in the anti-blur sub tab adjust the amount schedule to 0.05 depth warping and fov sub tab for those with Nvidia graphics cards no action needed here but if you're using an AMD card disable use depth warping that wraps up the keyframes tab now let's head back to the main tabs moving to the prompts tab paste your prompt from the image to image tab making sure to use the right format don't forget to place your negative prompts in the negative prompt section in the init tab under image init set the strength to zero for video in it paste the path to your original video copy the path by right-clicking and selecting copy as path on Windows 10 use shift plus right click ensure to remove double quotes from the start and end of the path also enable overwrite extracted frames in the control net tab enable the first control net unit and input the settings used in the image to image control net units now enable the second control net unit and replicate the settings make sure to copy the original video path into the control net input video path for both control net units now moving to the hybrid video tab set hybrid settings to before motion enable generate input frames select hybrid motion as Optical flow adjust the flow method to disk medium finally under hybrid schedules set comp Alpha schedule to 1. last but not least in the output tab match the frames per second with your original video to simplify your workflow you can save these configurations into a text file for future use in stable diffusion just assign a name to your settings file and select save settings this action will generate a text file within your stable diffusion web UI directory when you're ready to revisit these settings copy the path of the settings file and paste it in the settings file section then hit load all settings to seamlessly restore your configurations when starting up stable diffusion now let's initiate the process click the generate button to set to Forum in motion keep in mind that the duration of this process can vary based on the capabilities of your graphics card as the transformation takes place your original video will gradually evolve into a new masterpiece infused with your chosen effects while we're waiting for your video magic to come to life why not take a moment to show some love if you're enjoying what you're seeing consider hitting that subscribe button and giving this video a thumbs up and guess what if you're hungry for more detailed and sites I've got you covered you can find a comprehensive tutorial on my website so let's keep the excitement going and make sure you're part of our awesome community foreign

Channel: Next Diffusion

Views: 27,423

Keywords: stable diffusion, ai, midjourney, ai image generation, ai video generation, stable diffusion tutorial, video to video, vid2vid, deforum, controlnet, transform videos, warpfusion, deforum video

Id: rytoKTs--Y4

Length: 7min 55sec (475 seconds)

Published: Mon Aug 14 2023