Most CONSISTENT AI Animations | Automatic1111 + EBSynth

what is up my friend today I'm going to show you a new animation technique utilizing the power of AI technology now this is a very complex way to get a videoo video animation and out of all the methods I've seen thus far the results are by far the cleanest and most consistent for this method I'm going to be using stable diffusion with the automatic 1111 user interface EB synth Premiere Pro and topaz video AI as some of the these softwares are not free to use you can also do this method for free using Da Vinci resolve other than that all you'll need is a video to begin now I would recommend keeping it short and at a 1:1 aspect ratio now that we have that out of the way let's Jump Right In all right first we need a video so I clicked on the wrong so I I have really did me dirty on the pre-selected thumbnail there but this is what I got you want to know something I have always wanted to be an elf man to keep things simple I made my video a one: one aspect ratio I would recommend using the same parameters a one:1 aspect ratio so you want a square and then make it 512 x 512 for this method to really work especially if you have have a video card with low vram I have my video here now we are going to need to split this video into a bunch of frames to work with so I will show you two ways so I'm going to use Premiere Pro for the first method this is my preferred method personally so we're going to select the video of me with my gaping mouth and then once it's loaded in go to sequence sequence settings and again we need this at a 512 x 512 or else it's going to take a lot of computational resources to actually work on the animation so once it does that it's going to make my nose even bigger then just go to effect controls and then just fit to frame about right right yeah yeah right there so go to export and then make sure you change your source file that you want the frames to export in make sure the format is in PNG then for the file name I'm going to do tie and then I'm going to do an underscore so that way it'll name every frame in numerical order three digits and then export now your Source frames should look like this what if you don't have Premiere Pro well you just go over to our trusty website easy GIF then they have a video resizer so we can just plug our source video into this upload it then we will get some something it broke so then we just go to our trusty video Candy and upload the sour SCE video here and my Internet is so once you have it uploaded just set the dimensions to 512 5512 voila and Export and just download it now we are ready to break this down into frames in the description online converter video to jpeg so you just choose file and then yeah just drop in your Source video and then just click convert and then once it is completed you just download it and it will come in a zip file so now you have all your beautiful frames right there then you can just extract this to whatever folder you want all right and just like that we are ready to generate right no we're not so the next step is to pick out key frames and pack it into a Sprite sheet some of you may be wondering what the that even is this is a Sprite sheet it's like a tiled image essentially so we're going to be using the free Sprite sheet Packer again the link is in the description So within this folder I'm going to create a new folder folder called Source frames and now we're going to put all of those frames into this folder and now we're going to make another folder and we're going to name this one key frames so now we're going to pick images for our Sprite sheet as our key frames and we want the key frames to be frames that show a lot of different different differentiation thing that's a word we're going to use the first frame so the best way to do it if you don't know is to hold control click so I like so that one let's do 30 and do one with my head turned so the reason I'm doing this one is because my head is turned and then let's do one where my eyes are looking at the viewer while my head is turned so we'll do 46 then we'll do one while my head is slightly turned and my mouth is open so we'll do that one for for this video that's showing somewhere on the screen I used four key frames but for this video since it's a bit longer and I have more movement I'm thinking about doing anywhere from 6 to 8 see how many frames we have so I'll do either six or eight just so they fit on the Sprite sheet evenly so I have 1 2 3 4 five that classic one six do s and eight 1 2 3 4 5 6 7 8 might need nine let's just do nine so let's do one where my mouth is still kind of I guess that one should work so we'll do that one that way we have nine so I'm going to do nine key frames you can do four it's probably easier with four I actually haven't done it with nine but I know it's possible feeling a bit risky so then we're just going to copy those contrl C or command C and then we're just going to paste those in the key frames folder 1 2 3 4 5 6 7 8 n why do I have 10 do I know how to count 1 2 3 4 5 6 7 8 9 yeah I have 10 frames so I'm going to get rid of this one because it's almost identical to that I don't know what happened there but so for for my Sprite sheet I'm going to have three rows of three so now go to the Sprite sheet Packer make sure you clear it because there's some weird art there initially make sure these settings are the same as mine just Json hash compact zero padding and let's drag our images to the Sprite sheet so just like that that should work pretty nicely so hopefully you have an idea of where what direction this is going so now we're going to download this as a PNG then I'm just going to add the Sprite sheet to my tutorial folder so if you're still following along first off congratulations now we can generate so we're actually going to be doing this all in the text to image tab make sure you have control net so load your Sprite sheet into here make sure you enable it Pixel Perfect go to line art for the control type go to realistic and then make sure the model is at line art for the second control net again let's load in our Sprite sheet and and this one I want to do open pose this I have found to be very important to match the facial expressions that you make assuming you're doing video of yourself if you're following along to this tutorial but if you're not doing a face you don't really obviously won't need this but if you're doing a body then yeah you'll probably benefit from using open pose now for the last control net this one is pretty optional so again load up our Sprite sheet and this one we're just going to use reference now depending on the results you get you can play with the control weight so I'm going to put mine at .5 because it's already going to use the line art but I want it I want it to get a lot of my skin tone so I want it to like really look like me but an elf a dark elf I guess if you want to label me someone's fire alarm is going off so hopefully you can't hear that another optional thing that you can do is enable a detailer so it makes the result of your face more proportionate again if you're using a face for your reference but for this tutorial we're using a face and then I like to use the open pose model in a detailer as well and yeah a lot a lot going on I know next we want to set the image size parameters so since my Sprite sheet is three columns of 512 and three rows of 512 so I'm going to do some math I'm going to need 15 36 1536 I think cool and with those parameters set all we need is a prompt so for this I'm going to use the Tyrant prompt generator because I want a highquality prompt in a matter of seconds so I want a realistic elf man cool and that that looks good so we're going to copy that and then we're going to paste it here since I'm using the dream shaper model as per usual bad dream fast negative V2 and now we are ready to generate that looks pretty good this took me a few iterations this I can work with and now I played with a detailer I actually took a detailer off because it was making my face a little too soft looking it was leaving out a lot of details then I also turned off the reference so these are the results that I got so depending on what you're looking for I want a more detailed version of myself but if you're looking for a more stylized version then you can kind of go off of these previews here to get what you're looking for but ideally I have the line art and the open pose these settings here I have line art set to 85 then I turned off again I turned off the a tailor so we are now ready to cut these up and animate them so now if you go to the description yet again you will see the Sprite cutter at easy.com so I added my Sprite sheet to the folder that we're working with you don't really have to but I like to stay organized so I'm just going to drag this into here and then click the upload button you can see it is now added here so if you go to cutting method I will do by tile size and then we just want 512 by 512 then PNG output format and then cut now just make sure that your images look good I've tried to use this option the by number of columns SL rows and it completely butchered it definitely recommend just using tile size then when all looks good just download the frames as a zip file and now I'm going to create a new folder in our folder I'm going to call this one AI key frames so now we can just extract these into our folder that we just created and then extract so now we get to do the hardest part of this whole process and that is renaming the key frames to match up to the ones that we created that way EB synth will know which order the frames will be animated in and where to apply the stylization so just click on the frame F2 copy then go to the frame that matches that F2 paste do this with all the frames and make make sure you name them correctly and make sure you rename the right frames so now you just drag the file with all of the frames from the video that we chopped up put that in the video section here and then for the key frames we want to use the ones that we styled so the AI generated ones and then just put that in key frames and now you should see something similar to this it should start from Key frame zero and end at the final frame in our sequence which for me is 238 so the default settings on this are pretty much what you want I personally keep everything as is the mapping essentially is how close you want it to follow the source video which 10 is like I said very good def flicker pretty self-explanatory if you lower this to like a 2 or A5 it's going to really you're going to notice a lot of inconsistencies so one or two is usually where you want to be then diversity some people lower this I personally keep it around 3500 I don't really touch it not exactly sure what it does I believe this makes it stray away from the source video a little bit so yeah I usually keep it default no no reason to play with that and now we are ready to run all so once we have all of that rendered you should see something similar to this you should have all these out folders with the rendered frames inside of them and yikes I noticed that the first couple frames in the first folder are actually scratched that every single frame in this first folder is pretty garbage I'm just holding the directional key to scrub through all the frames all right so I noticed that in this first batch in this output folder it was rendering 0 to 100 so you want to keep the amount of frames that it's rendering per batch relatively small that way each key frame that it's rendering it takes that key frame and then applies that to the next sequence so for example it's taking the reference we have at zero then it's applying that style to 29 frames and it's much easier for the style to be applied to a smaller batch number because there's less movement going on as opposed to 0 to 100 frames so the reason that it was so destroyed is because it was it was taking that one key frame and applying the style to to frames that are very different from the source key frame if that makes sense hopefully that makes sense so this is where we kind of have to nitpick certain spots in our animation and regenerate if need be so let's open up Premier Pro and get to the composition all right so how we're going to do this you can go to file import navigate to the file we want to import and then click on the first one and then make sure the image sequence is checked and then open so if you add in all of the clips as I have done now one thing that will help you if you go to sequence sequence settings and under display format change this to frames you will be able to see the frames in the timeline so for example I have some inconsistencies here so depending on your animation and how many key frames you used you may have some issues like I'm having so first off you can see like the eyes start to warp and that is because I did not use a key frame like that I had a key frame where my head was turned but my eyes were in a different spot so to keep this simple instead of having to go in and regenerate all these frames when I recorded this I should have kept in mind that I shouldn't have that much motion in because it's going to make me have to do more to get it really consistent so I'm just going to trim off the beginning so it looks like this this looks much better we still have a lot of inconsistencies with the lighting but as far as any deformations this is passible so one thing I can do to help alleviate some of this inconsistency with the light is throw on the source video then just play with the opacity now this only helps out a little bit so stuff like this is another reason why I love topaz so I think this looks good as far as composition goes all my Clips are layered just the way they should so now we just need to fix this lighting issue so I have it loaded into topaz now this is where we're going to clean it up so I'm going to upscale it to a 1024x 1024 can do an a 60fps frame interpolation then to really clean up the lighting issue we're going to use stabilization then we're going to apply jittery motion with one pad and then we're just going to turn that strength all the way up now sometimes this has habit of blurring things out so we're going to start at 100 and if it Alters anything that we don't want we're just going to kind of tone it down from there then we have the frame interpolation model then the enhancement model as well then I'm going to export this only takes a couple seconds now let's see what that looks [Music] like I'm going to increase the stabilization to see if I can clean this up a little a little bit more cuz we're still getting the shiftiness from the lighting so we're going to go to two passes I'm going to go to let's do 69 and then just going to preview that see if this helped it is a lot cleaner obviously it's not going to make it perfect but it definitely stabilizes it makes it blend a little bit better there is a lot of movement in this video compared to my previous one my first one was much easier to create and the result is a lot cleaner but wanted to introduce a little bit more movement so I don't think this turned out too bad at all it saves so much time and resources topaz is definitely one of my favorite AI tools so I actually reached out to Topaz to get an affiliate link cuz I love this tool and I want to help them out so if you want to help me out and you're interested in buying this my link is down below so if you made it this far congratulations I hope you learned something and if you want to stay up to date on all things generative AI art I would definitely consider subscribing and at the time of recording this we just hit 1,000 subscribers and if you're interested in joying the Tyrant Empire private Community focused on bettering themselves in all aspects whether it be generative AI Fitness mindset business check the link in the description below to join the Tyrant Empire for free and with that said my friends I am out of here till next time keep conquering

Channel: Ty The Tyrant

Views: 8,176

Keywords: tyrant, empire, tyrantempire, rampayj, ai, artificialintelligence, art, midjourney, stablediffusion, automatic1111, leonardoai, dalle, chatgpt, fitness, selfimprovement, selfhelp, learning, gaming, apexlegends, leagueoflegends, callofduty, dating, women, redpill, entreprenuer, freelance, freelancing, money, business, memes, streaming, tyrin, music, musicproduction, dj, howto, makemoney, america, usa, lamborghini, ferrari, bugatti, honda, funny, muscles, abs, sixpack, 6pack, diet, food, polymath, books, reading, god

Id: KFKnyGwY2D4

Length: 17min 17sec (1037 seconds)

Published: Sat Nov 04 2023