I Can't Tell it's AI! - AI Animator BLOWS my mind!

man oh man am I excited for today's video we have a very nice AI breakthrough to talk about today this my friends is ton crafter the future of AI animation as I see it right now first things first here I do want to show you guys some examples this is going to look like magic what we do is we have one starting frame and then we have one ending frame and the AI will take those two frames with a prompt and interpolate the animation so yeah like I said we're just feeding it two images and then this is the quality of video that it can generate or essentially animate out of just having those two frames check that out I mean it looks like actual anime footage right I mean if I just showed you this would you believe that it is mostly AI generated I mean you can see maybe a little bit of weirdness up here in the corner but the hair is looking quite good the background is so stable it looks like actual animated anime cartoon whatever you want to call it it's so darn cool and this is like magic to me and by the way folks this gets even better because toun crafter right here as you can see on GitHub is actually under the Apache 2.0 license which means that it is indeed open source meaning you can download it you can modify it you can try to run it on your own machine and of course by no means is it perfect it's just research but it's research that's going to benefit the entire community at large and allow us to essentially build off of the magic that we have contained here and make it even better it's not limited or controlled by One Singular company or entity all right so let's take a look at this second example obviously something has been stamped here our first image is that Stamper tool just slightly lifted off off of the page and then the second image has it almost entirely out of the frame so we're just looking for this to move and nothing else inside of the frame and if we play that's exactly what we get and it's very very realistic looking it looks like it was actually animated well it looks like it was int there's not too much AI weirdness going on at all and honestly guys I think this is very telling for the capabilities of this tun crafter AI animator we'll call it you can see that the obscured stamp There is almost perfectly clear I mean there's a little bit of weirdness around the edges as it lifts up but it's pretty watchable I mean if I saw that in you know an actual anime I really wouldn't think too much of it not to mention as the tool gets obscured it doesn't warp or anything weird like that very very very impressive and as these scenes get more and more Dynamic this tool gets even more impressive so you can see we've got another scene with something shining and glimmering and again folks we see more of the same it absolutely can tell what's going on here there's something shining right from the center of the screen and all the Sparks are moving in an outwards Direction it's really quite incredible that it's able to pick up on really subtle nuances like that you can see this one's very very much difficult for this Aon ey as the person's face has a little bit of warping over there the hand as it reaches forward to grab it is warping as well but again it was able to interpret all of those frames in between where we know for sure that that person is moving in and reaching towards that glowing object it's really absolutely incredible and I wish that they had a direct comparison to the original scenes and I'm no anime nerd so I don't know what all of these different animes are or where to find their original scenes but tell me if it looks you know correct to the original scene because I think that would be quite impressive and again folks we have yet another very impressive one this is kind of a wide openen far away shot and we can see this person who is clearly walking a little bit in this direction so is it able to animate the person walking which is typically for those of you who don't know very difficult for AI video generators in general the only one that I've really seen able to pull it off is Sora and again this isn't exactly like Sora it's not the same type technology by any means but as you can see it absolutely can animate someone walking and that to me is probably the most impressive example here it it has this understanding that that's what it looks like when a person walks that's when one foot goes in front of the other in this animated factor maybe because it's animation we are not seeing as many of the details messed up or we're just more accepting of any details that look a little bit screwed up because with something like Sora obviously with photorealism we can really tell when something's not real but with a the cartoon not supposed to be real so if it's not exactly perfect doesn't really bother us that might be a little bit of what's going on today but regardless guys I'm just so impressed it's like magic to me and as you're watching all of these little demos folks you might be wondering you know can I use this myself and the answer is absolutely we do have a demo here on Tomb crafter and you can see that yeah I uploaded some sity silly photos of myself and actually tried to get it to animate photos of me and it kind of worked kind of didn't work but we will try it with some actual animation but yeah there's a demo that you can absolutely use for to crafter that I'm going to link down below and there's some different settings there's some examples as well before we get into actually using this thing ourselves though I do want to dive a little bit deeper into toun crafter for this one they actually do indeed have a project page that kind of lays everything out a little bit more simply including this little YouTube video let's take a look at that first so yeah this is a comparison between all of these and if you play it out you can see that this is super garbled and mushed and this one is nice clear and honestly almost impossible to tell that it is AI and as this continues you know we have the walking animation again this Ace AI which I haven't seen before to be honest completely gets Blown Away by toon crafter in terms of its ability to actually animate things as we expect to see them and wow check that out that's actually some impressive animation with the lighting and everything it's able to really understand where things are supposed to go yeah with the train coming there wow that's impressive I'm just so shocked and impressed in general by the quality of the animation more or less like I could absolutely see this replacing traditional anime animation I mean that stuff's not expensive and if you can just interpolate two frames like how easy would it be to create anime series and actually animate them properly could save a lot of budget a lot of time and I mean also allow people who are just pretty talented at creating art at home to actually animate things very very easily okay as we watch further we can see this guy hunch down a little bit of weirdness with his arm so yeah this is called sparse sketch guidance and as you'll see here essentially allows you to input a sketch here to help guide the AI in the direction that it needs to go in to properly animate things and you can see they even try it without a traditional cartoon here I believe this is footage from Final Fantasy please correct me if I'm wrong but you can see that there is some sketch guidance here to make this all possible but the animation just turns out so realistic and it looks so darn cool this extra guidance is absolutely necessary to achieve the highest quality results physically possible and obviously the whole reason that these sketches are able to help in the first place is because they are a wider outline of the movement at hand it allows the AI to understand better what is going on exactly in the scene just as it helps you kind of make things out a little bit better when there're you know these rough sketches without some of the more colors and details distracting your brain I mean they kind of do in a way work like a real human understanding things visually so we can also colorize sketches here as well which is really really incredible I mean check that out man that looks like a fully fledged fully animated piece there the lighting is really incredible that's actually Mega impressive man I mean the animation isn't perfect in all of these but wow this just absolutely mind-blowing technology okay and again folks we do have a few more examples if you want to take a look at here's a famous one from Spirited Away you can see it actually interestingly enough interpreted her as blinking during this scene which is pretty darn cool so I like to see that obviously also picked up on the fact that she is on a Ledge and the wind should be blowing her clothes around and her hair and I have seen Spirited Away I can say that's quite realistic I think to the original maybe a little bit more slow down maybe not as rapid movement in terms of the wind but impressive nonetheless we've also got the this train here blowing through the water and again I think the anime style is just more simple animation in some way so it's good or it's a good match for these AIS quite often and I mean that water one's pretty much flawless I mean it just looks like it's crashing through the waves as we would expect we've got some fire here again you can see some of the weaknesses here the fire looks sort of like it's moving in slow motion but the person leaning over looks quite good again you can see the witch coming through the floor right here and that looks pretty stellar as well well it really depends on the scene for how good things look but it looks like wow even the 3D animated scenes are looking quite incredible as it moves through that passageway that hallway again this scene where she's just sort of looking around and blinking looks really incredible as well it's incredible it can tell that she has eyes and that they need to Blink while the scene is going on you can see again with the comparison uh their's is just quite a bit better than all of the other methods here it's it's the most realistic it's the most coherent and I'd say that this is one of the closest AI video generators or AI animators that I've seen for Prime Time essentially ready to be implemented into animating Studio applications or video editors something like that it's almost ready for prime time you can see with the car here it actually does struggle quite a bit but in comparison to the others it's way way better looking it gets those tail lights a lot more consistent here's another Stellar example of how good their version is in comparison to all of the others like I said pretty much ready for prime time I mean by no means perfect but definitely usable I think for a number of real world cases this one really impressed me as well with the whole globe thing moving down I mean that's just absolutely incredible and again look at this one look at this example with the kics moving on the ground you pretty much only see the kics move properly with the ton crafter demo here at least for this frog and I think that's very Talent telling about the way that this AI Works under the hood and how well it's able to interpret the movement that should be happening inside of these scenes and we also see that with the tea pouring where everything else just looks kind of stable and still theirs actually looks like real anime animation I mean it's absolutely mind-blowing the colorization is also really really incredible I'm not going to dive too deep into this one but you can see you just give it some reference images that are colored you give it some rough sketches and then it just fully animates them for you it's quite incredible that it's able to do this and that this works so well and so stable it's really able to just fill in most of the gaps for the human animator here and it can make the animation process as a whole a lot faster and also easier to get into I mean animation is something that I am pretty passionate about I used to do the little handdrawn Flipnote animations and I got to say this looks like an incredible technology for people who want to learn to create their own stories and create their own animations in a much simp simp and easier way I have so many questions I really want to put this thing to the test today so there's a lot of demos here that you guys can take a peek at I'm not going to show off every single thing that is on this demo page I want to actually get into using this thing ourselves so let's go ahead and take a look oh and by the way folks as always all the links are going to be down in the description below from the actual showcase page to the GitHub where you can download the code and the weights to the actual online demo for those of you who don't know how to run it locally so yeah I gave it two similar looking images and they're pretty ridiculous one of me just kind of standing with a funny face and then I tried it and yeah I got to say obviously didn't really come out too great at all I'm just sort of warping around but it's not really designed for this use case so I'm not going to hold it against it what I will do is try to do some animations of my own so let's go ahead and remove these input images and folks let's break out paint good oldfashioned Microsoft Paint all right so right here guys I'm going to draw some sort of a rough you know cartoon character head we'll give him two eyeballs and you know some weird looking nose and yeah he's just your Chum he's a happy guy really really basic image so now folks what I did over here on the side is I duplicated the layer so now I have two of the same exact image and on the second layer I'm just going to change the eyes the nose and the mouth and we'll see how this AI interprets that and handles it so let's go ahead and erase this stuff just like that so those are where our eyes are now I'm going to give this guy a tongue he's going to be sticking his tongue out and you know what let's make his tongue red for the sake of the cartoon a little bit more detail on the tongue I'm going to put his eyebrows up and we'll give him his eyes here and of course his nose so we're starting with this image and we want it to move to this image all right we got our two images in here I'm really excited to see how this works they do have a prompt here so I'm going to go ahead and say animated character sticks his tongue out all right let's go ahead and click generate now the process does take about a minute but honestly in terms of video generation that's not too bad especially for a demo that's on hugging face and is completely free okay folks our animation here is done and there you go okay that is not too bad man that is pretty incredible as you can see it did all of the different steps here he closes his mouth a little bit he starts to open it his eyes move up a little bit and they move up a little bit more and he sticks his mouth out and his tongue comes right out just like that I mean that is so incredible to me it blows my mind I mean how many of you would be able to animate this that well it's not perfect by any means but it's definitely a hell of a lot better than nothing I would love to see this integrated into an app where we can just draw scribbles and doodles on our PCS or our phones or something and then create animations quickly and easily okay let's mess with the settings a little bit I'm going to turn the CFG scale down a little bit and I'm going to turn the FPS way up and by the way I just have to stop and say that the fact that this is open source is so great and so awesome because that means that this incredible technology is not just limited to you know one company or two companies it's out there for everybody so there will be free apps that use this kind of Technology there will be paid apps that also use it but are maybe a little bit more refined accessibility to for everybody at every different level is really the best thing for AI technology at least from what I seen so far all right so I U the FPS a bit and I think this looks maybe a little bit better but it's still like only interpolating a few of the frames I think because it's trying to do this anime method you can see like these two are different frames here but they're barely different from each other so maybe it's just another setting that I need to change let's try turning the sampling steps all the way down to one I'm just sort of playing around and messing around at this point to see how this thing maybe Works a little bit better under the hood okay so don't turn the sampling steps down all the way because it just turns your uh interpolation into a blurry mess let's try 25 by the way lowering the sampling steps absolutely makes your generation happen just a little bit faster so that's good to keep in mind I have no idea what ETA is so I'm going to turn that down to half and just see what happens all right with those settings oh okay all right I think the sampling steps might have been fine but I think that other setting we tweaked maybe screwed up the animation a little bit more so we'll leave ETA there you know leave all the settings more less the same maybe you can turn down sampling steps a little bit and get away with it probably depends on how detailed your images are I would love a version of this with like four image inputs so we can tell a longer story and have longer animations but again it's really important to note folks that this thing just just came out so the project launched on the 28th of May and the code and model weights were released on the 29th so there's going to be a change log there's going to be updates to this and I would love to see a more long form version because I think think that there is just so so much potential with technology like this all right let's try something else folks I'm willing to take this one step further I'm going to go here into idiogram Ai and do a 16x9 image tree falling over before and after photos I want to see if we can skip a few steps and just go directly to fully animating AI generated imagery okay and interesting results here for before and after the images looks somewhat similar this might kind of work let's give it a shot okay we have our before and after photos for the prompt tree withering and falling over let's give that a shot H okay something tells me that these two images are way too far apart to actually work because it just kind of cuts to them all right guys I think this one might actually work although the people are wearing two different sets of clothes all right folks let's see if this thing has any better luck okay well this is pretty interesting here it just cuts right to the shock face because I think they're just too different but then it actually does animate this guy kind of screaming which is pretty cool maybe the prompt has a lot more to do with this than I first thought all right maybe I was wrong because it just made him scream yet again finally folks I am going to go ahead and Link proper prompter examples down below that he's collected and these are honestly quite incredible and some of the most impressive things I've seen people do with toun crafter this one appears to be done with three frames and a comfy UI wrapper which is interesting so this gets pretty deep I'll definitely link this down below for those of you who are interested honestly what I am most excited about for this is not necessarily animators using it already I'm more excited for very creative people who don't necessarily know how to animate starting to tell and share their own animation stories by utilizing technology like this that's what's really exciting to me I really think that the democratization of these creative tools is going to bring forth a new era of creativity as a whole and something I've always been passionate about whether it be YouTube allowing anyone to upload videos online or something like I explained earlier with Flipnote which is this little app for the Nintendo DS that allowed you to animate things on a flip book I love the accessibility it means a lot to me and I'm so excited to see what toun crafter has in store for us in the future and what the rest of uh this year is going to look like for AI animation and AI video generation as a whole thanks so much for watching folks let me know what you think about this down in the comments below and I'll see you in the next one goodbye

Channel: MattVidPro AI

Views: 48,321

Keywords: mattvidpro, mattvidpro ai, ai news, ai animation, toon crafter

Id: ricylysRayw

Length: 19min 9sec (1149 seconds)

Published: Mon Jun 03 2024