Mastering Video to Video in ComfyUI's Stable Diffusion (Without Node Skills)

[Music] I recently posted a video and the results surprised me since there was no complex node knowledge involved the process simplifies how to turn real footage into instant style animations and only involves the use of two primary extensions to explain control net Trac the video footage outline while animate they generated the animation style prompt inner Reflections created the workflow which was then slightly Modified by consumption which I will make available in the description I will clear my canvas and load the workflow into cfy UI If you experience any missing noes go to the UI manager install missing custom noes click install for any notes that might show up here and restart cfy UI once again once this loads up everything is already set up and connected we have the input video animated noes prompt node and the control net node we will load any footage video into the load video node I have resized the original video to a vertical ratio so the original video is 4 seconds in duration playing at 25 frames if we come down here the original video is 4 seconds playing at 25 frames per second this will give us 100 frames so the frame load cap will be 100 frames as the video duration next I'll change the width to 57 6 and the height to 1024 which is a vertical aspect ratio down to the checkpoint Noe I'll keep this at hello 2D young also out select the vae to be klf8 animate and my decision is based on civit AI model requirements I will move next to the animate diff nodes uh down here everything should be set to default and only change the model if necessary if these models do not show up in your notes download them from the haging face page and place them into the proper directory in comfy UI from here we will now proceed to The Prompt notes put a good description of the original video as your prompt and based on my experience this is extremely necessary my prompt will be a female young model posing at the beach holding a surboard self model scenery in the prompt batch note the animation will be changing according to the description at each frame number because our duration is 100 frames earlier I'll keep mine at zero and 50 frames to be equal I also want to keep this all in a summer day Direction remember to match up your max frames to the original video duration which was 100 frames scrolling down to the negative prompt I'll would keep this at default but only add not safe for work from here we come to our control net nodes I will leave the default which is the line art model and also the processor node which is the line art processor I will use a save node instead of a preview node for the processor node you can then convert the frames into a video for visual demonstration this can be found in comfy UI under outputs once this is done if you don't see your models under your notes visit the haging face page to download the models for control net place them under your comfy UI directory in the control net folder refresh comfy UI and move to the K sampler I'll change this to 40 steps I'll leave everything else at default scroll down to the next note which is the Primitive I'll leave a seat at fixed and only change the increments by one if I don't like the generation scrolling to the video nodes I will delete the video node and and save images since I wouldn't be needing this rather I'm going to replace the video node by the VHS combine I'll double click search for VHS select VHS combine and I'll just connect this to the VA code to see our final video if you don't have this node go to the manager install custom nodes and make sure to search for VHS which was made by cinker Dink click install and be patient for this to be done close and close and back to the workflow you can refresh and I'm sure this will show up once you double click to search VHS under the combined note I'll change the frame ratees to match up to the initial video frame rate which was 25 frames I'll prefix the name to anything I want I'll change this to Ser model I'll leave the format at G and everything else at default default from here all we need to do is to click Q prompts to see our generation of everything we have so far this might take some time and be patient for the process okay so here we go just by changing the basic settings and prompts in this workflow we can see this incredible results from our original footage although these results are pretty amazing and if you enjoyed it you can take it a step further by controlling the movements of your animations which was explained in this video on the [Music] [Music] screen

Channel: goshnii AI

Views: 2,730

Keywords: ai video generator, video to animation, ai animation video, comfyui video, ai art video, animated gif video, ai video generation, generative ai video, beginner's guide to comfyui, beginner's guide to stable diffusion, ai video animation, ai video, how to install stable diffusion on windows, ai news, jboogx.creeative, animatediff, animatediff tutorial, ai video tutorial, ai dance videos, how to make ai videos, comfyui, comfy workflow, comfyui tutorial, vid2vid

Id: OF58t_7-rSE

Length: 6min 2sec (362 seconds)

Published: Thu Jan 18 2024