Did We Just Change Animation Forever... Again?

[Music] making a polished animation takes hundreds of people millions of dollars in huge Studios but what if I told you the clips you're watching right now were created with free software and just a couple of buddies working together [Music] and you might be saying Nico those are just a couple Clips there's no way you guys can do a whole episode of a cartoon you aren't even traditional animators well watch as we attempt to invent a new way of doing animation and if we pull it off you will see stories from small creators like us we've never seen before [Music] thank you [Music] all right so we are currently filming anime rocket processors too like filming an enemy aren't you supposed to be drawing an anime also you're probably wondering yourself why I'm dressed up like a little kid let me give you a quick background here so over the past year there's been an explosion machine learning for image generation and it's been kind of a Grassroots community-led thing rather than like a corporate development we took a bunch of that and experimented with it which culminated in a project that we made called animate rock paper scissors foreign film parody of anime but what was unique is that we used one of these machine learning image models called stable diffusion to convert real footage of ourselves on a green screen into cartoonified versions of ourselves and people really respond to this story and we were decently successful in translating that anime style the thing is we've always wanted to make a full episode of an anime not just a YouTube short but a whole episode with a full story and all that kind of stuff and we want to make that happen with anime rock paper scissors too I'm super excited we already did the hard part which is like the real acting of the voice acting so this is literally just us being puppets all day and playing around the first rock paper scissors while it was visually cool there's also a lot of technical shortcomings the faces get really wonky at Parts the style gets screwed up parts and it kind of looked more like a video filter than an actual proper cartoon and this time we wanted to push beyond that this time we need to transform these characters so to test if that's possible we are shooting the first scene of anime rock paper scissors too where we are looking at the princes when they are young children that means well either we film with kids or we use the promise in this new technology and we film ourselves and we turn ourselves into those characters as kids they say uh never film with animals or children so we're using machine learning to eliminate jobs from children I know I know people might get upset you know limiting those jobs for kids you know getting rid of that child employment but then you gotta do what you gotta do what we're filming on the green screen me in this costume it's not just gonna be the same thing but filtered I need to transform my my jawline needs to change this wig needs to look like real hair I need to transform into a character not just be redrawn with lines around my face we don't really know if any of this is going to really work this is all theoretical right now we actually haven't done it this is effectively our test if it works then the whole project is a green light and we're going to attempt to make a full episode of our very own anime I don't know that this is going to work with Nico with the beard we didn't we didn't do any tests on a bearded man as a boy my fear is that we're going to have a lot of weird AI artifacts on Nico's character from the beard and if we can't make that work then this day of shooting is kind of ruined and the video itself might not work so uh we'll see how that works out Nico this beard this beard issue issue so you're worried about like having a baby face have you seen the Adventure Time video that we did on Corridor I have not shaved since this video since 11 years ago look no no if it didn't work this beard was gonna go I'm more interested in making a good video than keeping my beard but I'm also interested in using the world's most advanced computer science to not have to shave all right so I'm going to start by showing you some still images converted using everything we've learned since the first video [Music] that is so stylistically different than like the first anime rocket processors like this looks like a cartoon but Nico so we take those settings and run a video of you through and apply some light compositing this is what we get look at that it worked and look you're ma like you can read the mouth movements it's like a little it's a little wobbly in the sense like it's a real motion of a person it's working and there's still some work left to do here but this is super promising like this is such a this is like a full body deep fake you know Nico it would have been easy to shave your beard but in not wanting to have a baby face you actually pushed us into a new frontier of Technology here hey what can I say I'm a great leader you know oh it worked my beard is gone this means we can green light the project it means we can go and move forward and make rock paper scissors too which is actually pretty daunting considering how huge this project is by our count the second rock paper scissors will have 400 shots the first one had 80. the second rock paper scissors have seven characters the first one had two second rock paper scissors has a whole bunch of environment changes the first one only had one so there's a lot of things we have to figure out so if you look at rock paper scissors one that was us experimenting it was us basically doing kind of like a fan film parody of anime specifically like 90s anime and also testing this technology to see if it would work and when we made anime rock paper scissors one we used frames from the movie Vampire Hunter D bloodlust to kind of create our style a lot of people have eaten criticisms at us for using the vampire 100d images now if you look specifically at the style of rock paper scissors one if you look at the frames they don't actually look 100 like vampire 100d bloodlust there's certainly like some DNA in there but one of the problems with rock paper scissors one is that it's just kind of a generic look it doesn't have its own style and it changes throughout the faces get drawn and interpreted a little bit differently if we're going to make our own episode of a cartoon that is our own thing we need to fix that you need to be able to see the characters clearly as a style that only exists in the rock paper scissors universe and it needs to be consistent so when you see the character in different scenes from different angles it still feels like it's the same character drawn by the same artist so with real cartoons they actually have a way to fix this there's something called a model sheet and basically they outline how the character should be drawn from different angles what the proportions are and this is given out to artists so they all can draw the character in a consistent way in theory if we do something similar for rock paper scissors too we should also be able to define a look for our characters that can then consistently be applied to them and for this I have a long time artist friend I'm going to reach out to and he's going to help us design the look for rock paper scissors too to create these character model sheets and if successful we should be able to Define our own style for this episode hi Josh it's me your longtime buddy Nico what were some of the key aspects to the style that you tried to develop but what would you say at your signature traits the things that I tended to exaggerate were cheekbones chins you got angular feature that looked like a great cheese on them um and obviously exaggerated eyes probably the most prominent of all brows everything was a rushed downward I'm not traditionally an anime artist so studying everything from the Empire Hunter D to dundum like getting to push those characteristics those personalities to an extreme that's so satisfying to do wow I'm just realizing I think it's still clear sketchbooks as an artist you have a computer that is looking at your drawings and learning how to apply them to video and how does it feel being on that spectrum of Technology the AI that we were we've been exploring is beautiful and exciting while also being a little unnerving and I feel like that's okay you guys were kind enough to solicit me and actually compensate me but for all the thousands of artists out there it's that are also getting sucked up by that jet engine of the AI but not being solicited or compensated or even told that it's happening my heart goes out to them the things that people are worried about like sound animation can be so easy to create we don't need animators anymore that's true if you want to tell the most generic diluted story that's already been told a thousand times if ever you wanted to be like original tell a unique story you need to feed it more original art and you need to just flood it because it needs to learn something new to tell a new story and if you don't do that it just starts to become kind of a generic amalgamation of all animations the best possible thing for it is that creatives descend on it ones that actually have moral compasses and passions and things they care about because this generation can provide the template that future Generations use when asking themselves like what's the right way to handle this hey all of you go check out level one art major on Instagram see some of Josh's work thanks for taking the time Josh I'll talk to you soon thank you have a good one so Nico you tell me that your friend Josh has made us some dope art yeah we have a bunch of cool pictures also some pictures of your character I have him right here actually yeah he's basically the handsome Squidward version of me yep I feel like eyes are the most critical thing that's the thing we have to kind of get right especially after the last one the eyes would be constantly changing colors yeah so we have our photo set our data set that we did for you so I took these photos as well as the art from Josh and I trained up a model and tried running it on these photos to turn these photos into basically the Josh's version of you oh whoa It's like his style but shrink wrapped to my proportion right the the way the lips are drawn is exactly the same like like the corners of the mouth like it's doing a little bit of a line for the philtrum but at the same time it also has the shading there the cleft chins coming through it's also worth noting that like it's taking his style and like bringing the costume to life through his style also like it's looking at how he's handling color and contrast and shape I mean you've got like the job Henry Cavill and all these yeah the drawings did I'll take it so this looks like it's pretty much ready to like run an entire video through it yeah um I tried I tried doing that I'm not gonna say anything just tell me what you think okay I mean it's like it's kind of coming through I think because there's so much flicker once you smooth it out it just kind of goes back to being that like neutral cartoon style you know it's pretty rough it's like if I pause it like this the frame's not that bad it's kind of getting Josh's eye shape it is getting the cleft chin but man when I hit play it is just like it is all over the place every frame is different the eyes are moving around it's it's like about where we were at like the last week of the first one and it was the best looking stuff we got for the original piece I don't know it doesn't feel like that big of a leap yeah it doesn't look like it doesn't but I don't want this to be just as good as the first one I want to be good I want to be really good it's a sequel it's got to be it's got to be a leap so Nico I feel like you've lost a bit of Hope but I sense there's still hope within you because if this wasn't working I mean we've spent a ton of time on this I feel like you'd be way more upset I don't know what we're going to do but there there's this thing called warp fusion which is created by this guy named Alex it was actually out when we were doing the first rock paper scissors and I tried to use it but it didn't have our reconstructed noise thing that we were using so our stuff would be warbly I had tried using it for our stuff I think our method was better but I just saw this animation posted like yesterday by this this guy named enigmatic e he has a YouTube channel and it's very similar to what we're doing here take a look at this oh whoa yeah the face reads the pose reads like the lines and the detail like there's a little bit of shift in Shimmer but like this is really cool like if there's an animation like this like this would be sick the details carry through even though it's such a huge departure from the original Dude it takes those details and carries them forward is that kind of what warp fusion is doing yeah like warp fusion it takes your image and it creates an optical flow pass basically tracks where things move and it morphs the previous frame using that Optical flow to where the the things would be in the current frame so it kind of pulls data from the last frame and warps it the current frame and then fills in like the gaps and things that need to be fixed but you get a lot of the data from the previous frame so rather than every frame being a new frame you're actually kind of cheating the system and getting it to look back and keep some temporal consistency going so kind of at a point where like our only option if we want to make this project is to try something new so I think I need to download warp fusion and learn how to use it damn dude that would be going back to the drawing board yeah essentially like we would have to we would have to redo everything we've done already we're like two months into this project almost and like we would have to throw out everything we've done we'd have to change our whole pipeline but if we have any hope of actually saving the project and making this work I mean from what I'm seeing warp fusion looks like it might work I mean from that animation alone it's worth a shot just it's so much python code and githubs I'm just I don't want to have to do it again I think you have to lock yourself in this room and you go you have to lock yourself in this room not leave until you have like usable results I'll do whatever it takes I'll do whatever it takes [Music] so Nico you seem very excited to show this to us so you know my buddy Josh he's been doing a whole like art style for this and we took that art style and we did some initial tests you look at it it's like yeah it's not awful but it's like on par with rock paper scissors it feels similar to yeah over this past weekend I started experimenting with warp fusion I actually spoke to Alex and asked him to implement reverse noise or reconstructed noise no one knows what to call it basically when you derive the noise from the image and he did so I decided to jump back in and I'll show you the results all right wow wow yeah that's amazing because that's a really tricky shot that's like a it's a big difference look how look how nice this one is [Music] wow wow this looks a thousand times better than everything else you've showed us it's my this is my halo moment when he first played Halo you know you're like this is it it can't get better like this looks so good anyway so I'm having that moment right here done all right we're done we're done that's the end so you figured it out and now we can do cartoons for real without any Jank is that are we ever gonna get to that point well the only thing that stops changing is us when we decide to stand still I still can't get over the fact that like brand new visual effects technique is in our hands as open source free software it's crazy we're running this on our our PCS if you have a decent gaming PC you can do this yourself the thing is I had to experiment and figure out my way through this and a bunch of people helps me on Discord and various communities it's amazing I figured I could help out a little bit just make a tutorial for anybody out there who wants to do what we're doing if you want to try doing this I have made a full tutorial series that's going on our website quarterdigital.com we cover everything from fine tuning model to using more fusion compositing and we also have a community challenge at the End plus a whole Discord server it's gonna be a good time if you want to learn how to do this Corridor digital.com there's a free trial you can check it out all right so we still have hundreds of shots to figure out so let's get back to it she used to yeah that's a lot of red yeah I mean look more Fusion works it's a generational leap from what we were doing with rock paper scissors one however this means starting over on all of our shots all this red is stuff that we have to do it's it's a shot that needs to be run through orb Fusion it needs to be composited some shots have multiple characters so you're running multiple characters through orb Fusion it's a lot of work imagine each one of these being like three to four hours dude this deserves an anime reaction seems like a hell of a lot of work it is a hell of a lot of work and there's no two ways about it but Nico why can't I just type in anime movie and then it comes out why do we got to do all this look you can type in anime movie and something comes out but no one cares what comes out when you type an anime movie people only care when you give it Direction an artistic intent and creativity we'll do it on there let's get it done so here's my solution I'm going to attempt to build build a online team of warp fusion artists I was on the warp fusion Discord a bunch of people were helping me out a bunch of people are making cool art a bunch of people are fans of corridor if we break out these shots maybe we can have like a group of bounty hunters grab two or three shots run them they can get paid for those shots I could reach out to kaitra she's been super helpful I can reach out to uh enigmatic e Eric you know the guy that basically taught me how to install warp fusion with his tutorials meanwhile we will in parallel make a compositing team here in the studio we know how to do that kind of stuff we have a couple cool animators we could hire kenson for example helped us out in the first rock paper scissors you're probably gonna need somebody to also help manage the edit maybe Matthias wants to come back in and help us out with that this one we just solved by talking to people and trying to find people who are talented at doing this stuff to help us out wait Nico are you saying you're going to hire artists ironic isn't it new technology doesn't make me want to slow down it doesn't make me go like oh I guess I don't need help anymore it makes me want to speed up it's like oh I can do even more now so let's do more yeah we can make bigger and better stuff now let's do it so we're all slowly making our way through the spreadsheet this is where the fun begins because we all get to be purely artists we get to take these plates and sort of break them apart and put them back together again combined with the backgrounds which Sam captures an Unreal Engine then converts into the anime style with stable diffusion and then we add 2D hand-drawn elements and even 3D Elements which we add in the composition to add that kind of final layer of sauce this creative process is accelerated greatly by our beastly workstation PCS provided by our friends over at Puget systems this is a subjective medium we filmed a plate of someone we put a filter on it and that's cool but the Artistry in making these shots sell whether they're like big impacts or big moments that comes from the creativity of constructing a dynamic shot after the fact you know we're just working with two plates here but like what else can you to it that really brings it to life like it's really fun being incredibly bombastic and over the top with these shots I think that's where the fun comes from for me anyways it's it's so funny sometimes thinking about how ridiculous the concept is these twins that play Rock Paper Scissors but I think it's also proof of how effective these techniques are despite the actual like plot of it this method allows us to get the most over-the-top intensity which helps you actually get invested in a Preposterous story and speaking of over-the-top intensity this wouldn't be an episode of anime without an anime theme song luckily we have a talented musician to provide the title track so my name is David Maxim mitich and I'm a composer and a producer from Belgrade Serbia how is it trying to write an anime theme song this was very very fun for me like there's so many animals that just like draw you in with their title track having a reference that you guys provided it was easy I would just like follow the concept that you guys already have when listening to it it still has your signature sound nice like I don't know there's something just about your your writing then we take the vocals and then have the vocalist who goes by the name shihori translate the lyrics as well she did a really good job too she did an awesome job you did the the rest of the music yourself right I did uh I got this incredible old synth module from like 1999 and it had so many of those like crazy early 90s synthesizer patches on it but it even has like the dues for Metal Gear Solid really that was like my big excitement with doing some of the music here putting in some of those like cheesy sounds to make it feel like a 90s anime once again dude I just I gotta thank you because the music is incredible it's a hit nice where do we go so hear more of your stuff Spotify my website all the regular places I do well have a great one bye bye all right it's midnight big moment all right you see that last little red thing there each red clip means there's notes and the last shot with notes has been dealt with this is it the last one oh where we did it is that all the shots every single shot in the piece is shippable we could upload this now and I would probably be okay with it and I think everybody would enjoy it a Project's never done until you give up but at least at this point we could upload it so everything from here on that is just us indulging in ourselves a little bit so we do have the final bonus scene at the very end I was thinking of a skit Dean tell me what you think so every Soldier is played by Jordan Allen in this piece they're all like clones of each other Jordan Allen is a soldier and Jordan Allen is a peasant walks by and the jail and Soldier says he looks up he's like there's gonna be a big battle in episode three but only if we get enough troops he's like well how do I help and he's like you can sign up at quarterdigital.com it's like okay and it just cuts the two of them in guard uniforms like hey this is pretty nice and that's the end of the piece hi so I don't know that's our little like what push to the website if you want to see an episode three that's how you do it also we have a full three-part tutorial that covers in detail every single step of this process so you can learn how to make cartoons using your camera and a green screen on your computer yourself it's surprisingly accessible like who would have thought Cutting Edge Tech like this would be in our hands for us to be able to play with there is something beautiful about something that would take the traditional Studio hundreds of people and years could now be done by five friends with a little time on their hands who may not have the tactical chops like the knowledge of construction method and rendering but they have a really satisfying story to tell and they want it to be heard that's gorgeous [Applause] [Music]

Channel: Corridor Crew

Views: 640,944

Keywords: anime, animation, animation process, anime art, anime artist, anime series, best anime series, creating animation, new animation tools, best animation tools, animation techniques, rock paper scissors, anime rock paper scissors, animation bts, bts, behind-the-scenes, stable diffusion, warp fusion, generative ai

Id: FQ6z90MuURM

Length: 23min 45sec (1425 seconds)

Published: Sun Aug 13 2023