Intense AI Video Maker (Stable WarpFusion Tutorial)

maybe you've seen some of these really cool AI generated videos these are actually video to video where it started as a normal video and then a whole bunch of cool AI effects were applied over the top of it to give it this look this is a video generated by Mr boofy here using this source of my videos that you see here as the original video these types of videos are really really cool but they're one of the more complicated styles of video to do with AI some other really good examples this guy James gerd over on Instagram has some amazing videos that use this style after seeing some of his videos he really inspired me to try to learn how to do this kind of thing you might have even seen some of his other videos because his videos of these marble Greek statues dancing have gone fairly viral lately here's another great example from enigmatic e who also has a really great YouTube channel on AI where he shows how he actually makes this style of video and then maybe you've seen on my Instagram or Twitter me sharing this video this was sort of my attempt to try to do what they were doing mine doesn't look nearly as clean as anybody else's that I just showed but I've been trying to figure it out and in this video I'm gonna walk you through the process of how this style of video is made now before we get too deep into the video I do want to warn you that this is not a free method there is actually some costs involved you're likely going to need to have an upgraded version of Google collab and the code to actually generate this style of video the person who created it actually only shares it with people who are members of his patreon so most of the videos that I just showed you were created with something called stable warp fusion and the person who developed that is this sxela I don't really know how that's pronounced maybe it's excella I'm not sure but if you support him on his patreon for ten dollars per month he gives access to his latest version of stable warp fusion so that anybody can download it and use it I'll make sure the link is in the description to where you can actually download it it's a pretty brilliant patreon strategy if you ask me because he's actually getting other people like me and like enigmatic e making videos that point people to his patreon but if you join the 10 per month tier you will get access to the download and once you remember you can see in his feed he's got stable warp fusion version 0.16 0.15 for this demonstration we're actually going to use 0.15 because as of this recording that's the most stable version from within his patreon we're going to download stable warp fusion version 0.15.5 i p y n b go ahead and click on that and we will download it to our computer here now the next thing you're going to need is a model so if there's a specific style that you're going for you're going to want to find an existing checkpoint model that's trained on that style and you can find those over at this URL here civetai.com this is a whole bunch of trained checkpoint models that work for stable diffusion but they're also the same models that you would use for stable warp fusion now just be warned when you visit this civit AI site there is a lot of not safe for work content I do have my filters turned on that you can see up here if you don't have the filters turned on you will definitely see some very adult content on here now the models that are on Civic AI are made available to download for free so there isn't a cost to actually grabbing one of the models that you like but let's say I want to animate something in this sort of cartoon style over here I can click into this model checkpoint see some examples of images that have been generated with this model and yes there will be some not safe for work images within here as well but I think this would be a fun model to test inside of stable warp fusion so now we need to go ahead and download it please keep in mind these are very large files this one is 5.28 gigabytes I'm going to go ahead and click download and it's going to take us a few minutes to download this one now another thing that you can look up if you want to add some extra special effects to your animation is we can come back to our home page of Civic AI here and we can actually come up to this filters button here and filtered by Laura's let's turn off the checkpoints and only show Laura's Laura's are sort of additional kind of add-ons that you can do in combination with your main checkpoint model so I can use that kind of cartoon animation checkpoint with Elora and the Laura will add some additional sort of effects to the video again there are a lot of not safe for work alloras so I'm keeping them blurred out but for example here's one called Bubble drip let's click into this and take a peek you can see it adds these sort of cool bubble effects to images could be interesting to see what it does to videos and these loras are actually quite a bit smaller than the model checkpoints so they should download a little bit quicker let's go ahead and download that one and while we're waiting for this stuff to download it would be a good idea to pick a source video since most of the videos we've seen are of people dancing I think that's a good way to demo what this is capable of so let's find a video of somebody dancing I have an account over at motion array so I'm going to go ahead and try to find a stock video video of somebody dancing you can really get the source video from anywhere but I'm using a stock video just to be safe to know that I have the right to use this video so here's an interesting video that will probably look fairly interesting once it's ready so let's go ahead and use this one now the video is 207 megabytes and it's an mov file I'm actually going to shrink it down a little bit make it smaller of a video file just because it'll be easier to work with so to do that I'm going to quickly pull this video into DaVinci Resolve the whole thing is 24 seconds I think I want it to just be about 10 seconds so I'm going to cut off a little off the front so I got it down to about 12 seconds this is the video that I'm going to work with here they're doing their little dance and they kind of all come together they do this little like wave motion and then it ends right there so I think that's a pretty good little clip to use I'm going to render this out in MP4 this will shrink it down so it won't be nearly as large of a file size and just to make sure the process is fast and smooth I'm actually going to bring the resolution down to 1280 by 720 and we'll render it out like that now if I show you this file you can see it's now down to four megabytes instead of 200 megabytes all right so we've got our checkpoint we've got a Laura that we can add into the mix we've got our notebook file that we can use with Google collab now it's time to open up Google collab and you can find Google collab over at collab.research.google.com and when you first go to this URL you'll see this screen here what you want to do is you want to click on the upload button over here and this stable warp fusion ipymb file that we grabbed earlier we're going to take that file and we're going to drag and drop it right in here and now this is what stable warp fusion looks like now I'm not a hundred percent positive but I'm fairly sure that the amount of compute power that this takes requires a Google collab Pro account so you can see I am on the pro account it is 9.99 per month so in order to follow this stable warp fusion process you do need to pay 10 on patreon to support the creator of stable warp fusion to get the downloadable notebook file also you're probably gonna need the Google collab Pro Plan which is also 10 bucks a month so just keep that in mind you kind of really gotta want to do a lot of these to make it worth your cost all right now that we're in the Google collab notebook it's going to look overwhelming there is a lot of settings that you can adjust and tweak here but luckily there's only a few that you really need to mess with so I'm going to walk you through them real quick so to start come up to run time up in the top menu and click change runtime type set the GPU type to a100 and then go ahead and click save this will make sure that you're running a higher end GPU that could handle the processing of stable warp fusion here then we're going to come up to the top right where it says connect and we're going to go ahead and click that and it's going to set up our GPU our Cloud GPU that is using behind the scenes so we're now connected and now we're going to scroll down a little bit we can scroll past everything here past the change log the credits and then you can see we have a section called setup now all the files that we're going to use the checkpoint file that we downloaded the Laura file that we downloaded the video know that we're going to have it processed all of that is going to run from within Google Drive so we need to go ahead and let it connect to our Google Drive so we're going to press play underneath this prepare folder section and then go ahead and click connect to Google Drive go ahead and connect it up and click allow and now we've got Google Drive turned on now if I open this little folder on the side that says files you should see a folder that says drive if we open this up we now have access to our Google drive here in the back end now we don't have to worry about any of these other play buttons yet there's a lot of steps here that we can just kind of skip the next thing we're going to do is under settings where it says to and then settings is we're going to change this batch name and this is going to be the name that is going to give the output file once it's done processing so let's go ahead and do animated dancers and then we need to make sure that the width and height is set to that of our video this one's currently set for a vertical view the video that I'm doing is actually flipped it's actually 1280 by 720. I'm going to go ahead and flip that here but just make sure it's set to the aspect ratio of the video that that you're actually working with hey we can skip past these settings here and then under video input settings it asks for this video init path this is actually where the video is going to be located that we're going to have it process so I created a folder inside of my Google Drive called warp fusion I'm going to go ahead and open this folder here and then this little future dance video that I created I'm just going to upload it into this folder all I have to do is drag it and drop it right into this warp fusion folder it's going to take a second here and now you can see it is uploaded inside this folder if I click these three dots here I can click copy path and then next to a video init path here I'm just going to replace this I'm going to click paste and you can see now I've got the path to that file that I just uploaded now this extract in frame here you can actually make your video process Faster by changing this to 2 if you'd like if you set it to two then what it's going to do is it's going to process every other frame instead of every single frame now this is sort of a personal choice if you have it processed every other frame it will do the processing quicker but it'll probably look a little bit choppier if you do every one frame it's going to process every single frame of your video so it'll look a little bit smoother in the final output but it will also take a little bit longer since I'm only doing a 12 second video here I'm going to extract every one frame it's probably going to take you know a good hour to process but I could live with that for this video so we're going to scroll past all of these settings down here it says store frames on Google Drive I'm going to go ahead and click that so that every single individual frame that it creates it's going to store each individual image inside of Google drive again there are a lot of settings here that if you are going to use warp fusion you can definitely play with them and test different stuff I'm just trying to make a real basic simple video for this tutorial so all of this stuff you can play with extract background mask it will try to rotoscope the main character that you're trying to generate and create a mask from that I'm not going to bother with that for this tutorial we're going to scroll down to this section that says Define SD plus K functions load model under load two I'm going to change this from CPU to GPU and then under model path this is where we're going to put that checkpoint that we downloaded from civit AI so under this warp fusion folder that I created in my Google Drive I'm going to open my models folder here you can see I have a handful of models already in here but I don't have this YIFY mix one that we just downloaded and I can't really just drag and drop it over into this browser here it's a little bit too large to do that so what I need to do is actually open up Google drive from within my computer so I'm going to go ahead and copy this file here I'm going to jump into my Google Drive go to my warp fusion folder that I created go into my models folder and I'm going to paste it using the file explorer on my computer here so now it's in there but it still needs to upload to Google Drive like you can see it says upload queued upload zero percent so it's going to take a little bit before this model is fully inside of my Google Drive and usable while we're at it might as well do the same thing with our Laura file so I grabbed this sy3 safe tensors that's the Laura file that we grabbed that adds those little you know bubble drips I'm going to copy this file here once again we'll jump to our Google Drive our warp fusion folder that we created and then I have a Laura folder I'm going to go ahead and paste that into the Laura folder here and once again check in Google Drive you can see it's going to take a little bit of time for that to process so while that's processing we can move on to some of the next steps here it's asking for a control net models directory so under this warp fusion folder I also created another folder called control net you would do that by just going to the warp fusion folder over here that we created earlier clicking on the three dots clicking new folder and then just creating one called control net I've already done that I've got my folder control net here so I'm just going to copy the path and then my control net models directory I'm just going to go ahead and replace that with the path that I've got there once my checkpoint file is done uploading I will replace that URL in a minute so we'll jump back to that in a second we can scroll down to our Laura and embedding paths here we already created our Laura directory earlier so I'm going to go ahead and click on these three dots copy the path and make sure our Laura directory is set properly I'm not using any cuss Testament beds for this one so I'm just going to leave that alone for right now okay so our model's finally been uploaded you can see it under our models folder here under yiffy mix so I'm going to click these three dots copy path and then under Define SD plus K functions next to model path I'm going to go ahead and paste this new model that we just downloaded from civet AI now we are actually pretty much ready to run this there's one last step we want to do we want to kind of give it a prompt to start so what we'll do is we'll scroll down until we come to this one that says non-gui here you can see we've got some cells hidden so we'll go ahead and view the cells that are hidden and if we scroll down a little more you can see you can set a seed here this can be any random number you want and then down here you can see this is where you would enter a prompt if you don't enter anything here it's by default just going to use this prompt that's already there a Disney style cartoon animation of robots dancing colorful highly detailed let's throw Pixar are in there and then we'll use that as our starting prompt now if we want to use that Laura remember we looked at this bubble drip Laura here there's a little bit of a code that you have to use to make it work the easiest way to learn how to use that is to come down to some of these examples of what other people have done you know click on the little info button on one of these images and see how they use the prompt here this prompt uses this little code sy3 so we can actually just copy this right here bring it over to our prompt and add it to the end of our prompt here paste that in it uses these little brackets Laura colon sy3 colon 0.5 this is a weighting so if I give it a weight of one it will use that Laura a little bit heavier if I give it a lower weight it'll be a little more subtle let's go ahead and leave this one on 0.5 to get a similar look to what we see here we're kind of blending some Styles here so I have no idea how this is going to turn out we'll leave our negative prompts the same that they already have here maybe I'll add disfigured ugly mutants you know some stuff like that that's going to possibly make it look a little funky and now all we have to do is is come up to the top here come to run time and just click run all and this is going to take some time we're going to watch it run all of these processes step by step some of them are going to take a few seconds this one only took four seconds some of them are going to take a few minutes as we move down a little bit you can start to see it processing some of the control net so you get a little idea of how it's outlining and seeing the original images okay so now we're finally to the part where it's running the portion that says do the run and what's really cool about this is in a minute here it's going to show us a frame it's going to give us a sort of example of what things are going to look like and if we don't really like the way it looks we can come back up to this new graphical user interface that it created for us here and tweak our prompts and tweak some of our settings so let's go ahead and scroll back down and you can see now it gave us a sample of what our first frame looks like so let's say I'm not really super happy with the way this is coming out I can go ahead and click stop right under do the run here now we can scroll back up here and under our new user interface we have a bunch of settings that we can start to play with let's change this to a Pixar Animation of robots dancing let's get rid of colorful let's make it in space stars in the background and let's go ahead and up our Laura effect up to 0.9 we can also click on our control net tab up here and change which control net models it's using I'm gonna go ahead and turn on depth and turn off the in paint and there's just a ton of settings that you can play with to get what you're looking for again since this is just a demo to show you how to do what we're doing I'm not going to play with every single step this is for you to get in and play around with once I've changed a few things I'm ready to go again I'm going to come down to under reference control net click play again come down to diffuse click play again on that one and then once again we're going to click play on do the run we'll wait a minute or so and it's going to give us another preview of what this new version is going to look like so here's our new version I like this version a little bit better you can see it's actually adding some more of these little bubbles from our Laura using those as kind of the Stars so I think this is a better version and I'm just going to let it run now you can see it's one percent it's estimating it's going to take roughly an hour and I think 41 minutes or so to run this so at this point just let it run we're gonna step away from the computer and just let this thing process for a bit all right so it's finished processing the run you could see it took pretty much about two hours exactly Here's the final screenshot it moved on to the process of creating the video and what it did is if I open my Google Drive now so I'm going to go ahead and just open it on my desktop it actually created a folder inside of my Google Drive called AI if I just double click into this AI there's a folder in there now called stable warp fusion this was all automatically created there's a new folder called images Out and because I've done some other videos in stable warp fusion there's other folders in here but this is the animated dancers one that we just created here so if I double click in here you can see every single frame from the video is loaded in in here you can actually click through them and see them all but what we really want to know is the video so if I double click on this folder that it created called video it actually rendered it all into a single video and we can watch the entire animation just like this we've got our dancing robots moving around doing their little thing the stars in the background the spaceships in the background now it is pretty flickery so one way you could sort of fix that is we can actually open up DaVinci Resolve drop our video into the timeline here go ahead and let it use the frame rate from the video you can see here's our whole video if I come over to the fusion tab here you've got this line that's got media in and media out if I click on this line here and hit Ctrl space bar it brings up this ability to select other tools and then I could type in D Flickr and bring in the D Flickr tool click add and now it added the D flicker now if I come up to the D Flickr settings we've got some options here for time lapse or fluorolite if I select fluorolite come back back to the timeline and watch the video it actually removes some of the flickering it's not perfect and then if we want to do kind of a comparison shot I could pull this original video into the timeline so you can see it here it should line up pretty perfectly with our existing video and then if I want to add that wipe effect I can essentially add a keyframe so let's come a few seconds in like this let's add a keyframe on our cropping I'll open up our crop down here we'll click on this little Diamond to add a keyframe for our crop and then if we slide a few frames forward add another keyframe and then bring our crop right all the way across like this now it will animate between them with that little sort of wipe effect and then if I want to do that again I can come up here click our little Diamond move a few frames forward click the diamond again and bring our crop all the way back and now we've got this effect here and then in a minute it's going to zoom back a little extra trick you can do if you want but let's go ahead and get rid of this one render this out with the extra D Flicker and see how it Compares here's the original version before we're adding D flicker now here's the final product it does still have quite a bit of flicker to it if I'm honest but I think that's only due to my own lack of knowledge on all of the various settings inside of stable warp fusion there's probably some settings that I could have tweaked I just am still learning stable warp fusion myself and I really think I probably could get it looking a little bit better with some more time so I'm looking forward to seeing what other people who watch this tutorial go and do with it now just for fun I also ran that same video through kyber I gave it the prompt of futuristic robot dancers that are dancing in space surrounded by stars and the style of Pixar animated cartoon and this is what it gave us here's the larger screen version of the animation it's definitely a lot less flickery than what I got out of stable warp fusion but also it doesn't quite have the same coolness factor to it doesn't have the same level of contrast and Chaos that was going both totally different styles of video but both pretty cool in their own right me personally I'm probably gonna to use kyber more often than I use stable warp fusion just because of the process being so simple and so quick but you definitely can get it much more dialed in and much more exactly what you're looking for using something like stable work Fusion because there's all sorts of different models that you can use from Civic AI there's all sorts of different loras that you can use to add little extra effects to it there are so many different variations and really so many different ways you can get it dialed in with stable wart Fusion it's kind of like the difference between using automatic 1111 and stable diffusion to get an image and get that image exactly what you're looking for through all of the little buttons and knobs and little settings that you can tweak versus mid-journey where you have a lot less options you can kind of prompt something and you sort of get what you get and it looks cool right so kyber is sort of like the mid-journey of creating this kind of videos where is you're going to get what you get out of it stable wart Fusion consider that more like the stable diffusion the automatic 11 11 where you can really really get it dialed in and get way closer to the exact exact video you're looking for so anybody who's been interested in stable work Fusion that's my tutorial on it I know this is probably not going to be one of my more popular videos on YouTube this is a very Niche one that only a handful of people probably really care about watching but it's one that I wanted to make because I'm learning stable work Fusion I want to get better at it so I wanted to show the process I want to show where I'm at with it right now I also see my YouTube channel as a little bit of a catalog of various tutorials along with the news and a stable wart Fusion tutorial was something that I just kind of wanted in my YouTube catalog once I get better at it I'll probably revisit it and show you even more cool tips and tricks that I learned as I get better with it myself thanks so much for tuning in I hope you enjoyed this video if you enjoy tutorials like this give this video a thumbs up and I'll make sure you see more videos like it in your YouTube feed also don't forget to check out Future tools where I show off all the latest cool AI tools all the latest AI news and join the free newsletter I'll keep you in the loop of all the latest AI news and tools and I'll send it every single Friday you can find all that over at futuretools.io thanks again for tuning in really really appreciate you I had fun with this one hopefully you enjoyed watching it see you guys later bye [Music] thank you

Channel: Matt Wolfe

Views: 43,346

Keywords: AI, Artificial Intelligence, FutureTools, Machine Learning, Deep Learning, Future Tools, stable warpfusion, warpfusion, disco diffuion, ai video, generative video, ai video generator, ai animation, stable diffusion, text to video, text to video ai, ai video creator, best ai video creator, ai tools, video editing, ai art, ai video editing, kaiber, ai generated, stable diffusion video, controlnet, ai video editor, stable diffusion animation, ai tutorial, ai video maker

Id: mVze7REhjCI

Length: 23min 27sec (1407 seconds)

Published: Mon Jul 03 2023