We Made a Better CGi Luke Skywalker

those dead eyes yeah pretty good there is a debate going on about how they did luke skywalker in the mandalorian so here's what we're going to do we're going to make a defect luke skywalker we're going to show you how we do it literally asking can we do better than a multi-billion dollar company called disney we might be able to [Music] the hardest thing to do in visual effects is to create a digital human being from nothing now spoiler alert if you're watching the mandalorian and you haven't finished season two maybe stop watching this video okay spoiler alert over luke skywalker showed up pretty neat kind of had a lot of mixed reviews as to how it looked and a lot of contention as to how it was done because disney has been very secret that's how they did it now in vfx artist react we do a lot of critiques but today we're going to put our money where our mouth is we're trying to create our own version of luke skywalker it's more realistic more emotive and more vibrant and lifelike than what disney has created we're going to be doing a deep fake what you're about to see is not really tom cruise someone took the computer and they made him say and do things he wouldn't say now watch and remember this is not tom cru ready run the bell yeah to get a little subscription plus action yeah and i'm having the time in my life it's such an honor oh it's an honor and a privilege don't forget that now i cannot overstate how big of a deal this is if we pull this off because it represents a fundamental shift in how visual effects are going to be done in the industry we're reaching a point where ai and machine learning can potentially let anyone create a shot that has before just been impossible so come along for the ride let's see if we do it alright guys we have some important very important business to discuss was luke skywalker deep faked in the final episode of the mandalorian there is a debate going on about how they did luke skywalker in the mandalorian i was fairly certain when i first saw it that it wasn't a deep fake because it looked more to me like moff tarkin or princess leia from rogue one very classic pipeline sort of cgi rather than experimental one person doing it on youtube deep fake style stuff now the thing is to do it for a tv show for like a 10 second scene at the end of an episode it's pretty nuts because you'd have to make 3d models you'd have to do face tracking you have to animation like so much work would have to go into it did you know that disney actually developed their own defect method and they released a whole technical paper and video about it what they did for luke there it does look like there's some element of deep fakery going on the challenge they were facing is that the person who is playing luke in the actual shot that they have to film with the camera is not mark hamill so they need to get mark hamill's performance stick that onto that person's face so i think they're dealing with like a double stack here they have an actor who then either has a cg like 3d head or mark hamill's actual like face being tracked on top of that and then you have a defect being applied on top of that now you're basically stuck with a double stepping kind of situation and that's where i'm wondering if we could surpass them you're literally asking can we do better than a multi-billion dollar company called disney we might be able to step one so we're gonna try to figure out how we can improve this shot so little things we can look for here the shadow is hand passing across the space like does it look defined or does it look blurry right the lighting is very soft and kind of an omnidirectional light compared to all the defined sharp lighting and the rest of the scenes it makes it kind of feel flat and washed out almost like airbrushed in how light interacts with stuff is what makes it feel real so if we can be hitting that scene with a lot of directional light we can really give things a sense of form and shape we have flickering lights and moving light sources and all these things that can just really help us sell that this is a real thing look how light interacts with it you can do this easily if it was fake look for those dead eyes yeah pretty good your eyes never just drift they always lock lock lock when he raises his head his eyes just smoothly move up like if you look at how his mouth moves that feels like keyframe targets to me doesn't that a little bit all these heads have very smooth movement they're like cute framed so smoothly got that s curve moving around whereas human heads like will like vigorously shake sometimes and we have kind of erratic movements in our heads people's expressions are connected like when i smile this happens this happens you know when i raise my eyebrows my mouth twitches the biggest takeaway from that scene is just how stiff the performance is period i feel like they're very locked in in a box and they can't turn his head too much or have them make big expressions because probably worried about how realistic it's going to look now if we can make our version of luke emotive and give smiles and you know eyes that are vibrant they look around that's all going to really help sell this is an actual character because as a visual effects artist you know one half of your job is actually constructing these elements that's going to go into a shot the other part is all the little tricks that are going to sell it as real it's classic visual trickery if we can do all that we can make a better luke skywalker we're going to use artificial intelligence we're going to use machine learning we're going to be doing a deep fake dpix going to let us put a luke skywalker mask on an actor it's basically a computer practicing to be a portraiture artist it's looking at somebody's face and it kind of paints a version of it it looks kind of blurry and weird but if it practices it over and over and over and retains its memory as to what worked well then you end up with a computer that can draw somebody's face pretty well and the magic of a deep fake comes from a computer learning to draw one person's face another person's face and then you switch the wires the nice thing about deep fig is all the stuff i talked about motion lighting expression even perspective you get all that without having to build out 3d models and deal with all the crazy physics simulations you need to do to get a face to look realistic the way we're gonna make the deep fake is well documented you can look up tutorials online we're just going to be doing the same thing now we have a couple tricks of our sleep first nvidia 30 90 gpu incredibly fast incredibly powerful a lot of vram so you can get some nice higher resolution deep fakes puget systems has been building us workstations here at corridor and that's gonna be our go-to machine for making this deep fake it's gonna let us get to a higher quality version luke skywalker higher resolution in a shorter amount of time than any other computer here in this office you're going to be seeing a cutting edge deep fake here so the key ingredient for a deep fake when you're drawing a face is you need to feed the computer examples in the face we need to inform the computer and how to draw luke skywalker so really the only version of luke skywalker we can use is the one that we see in return legitima so i'm going to enlist peter here be my teammate together we're going to try to gather as many high quality pictures of luke skywalker as possible every angle every pose and every lighting condition return of the jedi was released in the 80s it was still shot on film which can be great quality and some of these shots are great quality but a lot of them are just blurry as hell and that's causing problems because the images you feed in is the kind of image you're gonna get out then we're gonna have a blurry deep fake and so right now i'm trying to use these different ai sharpening tools and some of them are giving some wacky results oh my gosh here is a i sharpened blue so it's just scary we've gone and found a bunch of pictures of luke skywalker but now we need to find our actor who's going to wear the deep face who's going to play our luke skywalker because all the deep fake is is a mask everything else we gotta do for real still closer we can get to someone that looks like mark hamill from return of the jedi the better can i volunteer i don't want any part of this wait luke but i was going to go to tashi station to pick up some power converters there he is i've been waiting my whole life for this moment dean has the exact same chin by the time we're done with you they won't even recognize you man i was hoping to be a jetty oh yeah dude i probably look the most like baby yoda it's between you and sam well we found our luke skywalker and dean hughes all along who would have thought so now we know it's dean we need to capture pictures of dean so the computer can practice recreating dean's face remember it's about being able to look at and draw dean's face look at and draw luke's face and then cross the wires in the middle we're gonna film dean under a bunch of lighting conditions making a bunch of expressions from a bunch of different angles i definitely have enough data to start this training this is actually great this is like a lot of really good like lighting samples so now we have our dean images and we have our luke skywalker images and the computer is training on those practicing drawing dean practicing drawing luke it's practicing drawing luke on dean but this takes time and it totally depends on the quality of the pictures that we have so it's imperative that we have this defect running as much as possible is made to get this trained on a 30-90 is still probably going to take us about 10 days maybe a little bit less everything needs to be set up to be running all weekend because we can't afford to lose that time man look at this look at this dean dean you're you're starting to turn into mark hamill man look at you don't look at that one the most we can do right now is just step back and let it run [Music] oh hey i don't know if you know this but the guys are at the studio right now and they're doing something that has to do with the mandalorian season two now i haven't seen it yet and i don't want to know anything about it so i've been holed up here in my attic with one purpose and one purpose only raid chat alleges courtesy of today's sponsor that's right one thing we like to do around here at corridor is play little dungeons and dragons and if you like dungeons and dragons and you like mobile games well then you might also like great chatter legends too the thing that i like the best about it is that you can pick and choose how you want to play the game they have everything from straight up arena battles to campaigns to dungeon battles where you can team up with friends and actually go in with your buddies you don't need to wait on anybody to play but you can also play with people whenever you want another thing you can do is summon different champions and level them up as you play the game now the fastest way to level up a champion is through farming a campaign you grind xp and get loads of silver at the same time picking and choosing champions is some of the most fun that i have in the game and i got i get hey hey little buddy what look at that guy right there i mean he's one of the champions right there you could pick him you could level him up you can put him in an arena battle you can put him in a campaign it doesn't matter because rage had a legend has it all that's right they've got fights with noble knights and dark wizards they've got tough bosses they've got millions of players and best of all they've got years of content and regular updates so that whenever you jump on and play the game whether you download it now or whether you download it six months ago there will always be something new for you recently raids developers rolled out the biggest update ever the doom tower that's right it's a giant tower with 120 floors a bunch of secret challenge rooms and 12 seriously badass bosses to take on so if you're interested now is a really good time to try raid chat legends yeah what do you think you think that's good i don't know what to do with that do you live up here do you live up here so if you're interested hit the link in the description to redeem your free void champion an xp booster 50 gems some energy refills and an ancient shard also check out this champion bulwark you'll get for free it'll be a huge help probably in the doom tower get on that right now anyway look at the time it's time to get back to work i think the guys have passed through this mandalorian spoilers phase i'm going to check in with them the coast is clear from here hey honey yeah they do you think luke skywalker did you know damn it we've been talking a lot about what we're going to try to create here we're going to try to redo the scene the mandalorian now you may have seen like other deep fakes in line where they defeat that scene but they're just putting a deep fake on top of the scene that already exists but in this case we're starting from scratch we're going to have our own actor their own hair and makeup on a blue screen with our own cg background the entire thing is going to be us so what we learned with our tom cruise deep fake is that performance posture body language all these things your hair those all matter so dean is not an actor but he loves star wars so i try to get my hair like a little more looky all right i'm george lucas i want to see you walk into this room and i want to see you take your hood off with gravitas okay [Music] he needs to be the puppet master driving the expression you know if dean's smiling in a way that luke skywalker never smiles because mark campbell doesn't smile that way it doesn't matter how good the t-pick is it's not gonna match we're gonna go comas air make this happen if we pull this off we got our shot all right we're about to start filming our shots we're filming on a green screen we're setting up all the lighting like all the ways i talked about so i can bring this to life we're gonna get demon costume i'm gonna put him in the scene i'm gonna realize his lifelong dream bring your child to work day i made a whole shot list of all the lines how i want to do a performance around them i've been working on some effects with the lights to like simulate sparks and things like that you know at this point we just got to shoot it and see what happens all right all mysterious you just came into the smoke action does it occur to you that mark hamill himself might actually see this that might be the best moment in my life dean is more of a puppet master than he is an actor right now he needs to puppeteer our deep fake so for him to puppeteer effectively he needs to move his face like mark hamill does all right good work nice work good acting d got everything we need right on this guy uh it was the best day of my life so just gonna go back to work can i can i keep costume you can keep the costume on the shoe with dean went great it worked far better than i had any like hope for it to work the green lightsaber on his face the flickering sparks it all worked it's really cool but we have a bunch of problems oh look at that look at that look at that lighting look at that lifelike natural luke expression look at these dreams being realized luke's hair is definitely puffier though luke's got the helmet thing going on we need to shoot it again with the wig it's gonna be it's not a big deal dean's neck is longer his jaw's a little bit wider his shoulders are a little bit more slouched and dean's eyebrows are a little bit lower to his eyes so once you put luke skywalker on him it looks like luke's doing this the whole time and he's just walking around the neutral like frown expression and there's no way we could have known any of this until we did it we clearly need to go back reshoot it again but we have a lot more knowledge about what's going to make this look realistic we need to get the hair right the hair is not working you get the posture right when you get the body type right but the lighting no i'm going to do more of that we're taking the shot to the next level today jordan went and got us some wigs that's right that wasn't my real hair could you tell this is what we need to use if we ever want to do a nico fake is this freaking anchorman dude yeah the background looks really good it's too wavy this is too short on the side this part is not right yeah it's not gonna work i'm telling you dude the color's not that big of a deal we can always alter the color george is coming i can literally show him this picture in that wig i'm like dude can you cut this in that hairstyle absolutely is that the wig right there that's the wig all right we're looking for something like this okay totally by coincidence today we were doing haircuts before work started anyway because a lot of the dudes wanted haircuts because they haven't had them in a long time so george is actually going to help out and cut the wig into the closest luke skywalker style you can close very close the colors a little tricky but dean needs to put on some masks and i think the only way to do that is to just have him do some pull-ups and then we'll stuff them with some rags and stuff but i think you got to like get the base layer until failure till failure bigger all right let's do this let's put on the full cup it does it does shorten my neck a little bit it does if you haven't locked it in i just want to see if you want to hit the cut off it's looking good it looks awesome the wig huge help it's not perfect but it's way better dean's performance much better much more lighting we got his posture he looks way more regal you know we padded up those shoulders either way i'm really excited but here's what we got to do now the deep fake is doing its final final final little bit of training we need to composite in cg backgrounds which we are just like disregarding as a challenge like that's a challenge still pulling a good blue screen key and doing cg backgrounds is still hard how's it going here it's going really good dude it's friday this video is coming out on sunday and we're still working on it the reason we're still working on it is because we have seen the potential and we are doing our best to make this as good as we possibly can dude defects gotta be deep we're doing our compositing you know taking our new luke skywalker and putting him on our new background that we've created like i just took it for granted that we'd be able to you know have a 3d hallway behind luke with like lights flickering and all that kind of stuff like i'll figure that out later because you know i've done that before so peter's just been cranking through that as fast as he can yeah just taking the original shot from mandalorian building out the geometry based on what was in that scene i imported a frame from the show into this program called f-spy which is super useful you like specify some straight lines in the scene and then the software interprets that and gives you the exact camera position and you can just throw that into blender and then you have the scene from the movie perfectly lined up with your 3d view so then you can start dropping in blocks and panels and stuff and start lining it up so it matches perfectly so what you've got to do for the corridor crew channel is to subscribe if you don't subscribe my voice will go back to being normal and i'll be very sad but if you do subscribe i'll continue talking like this forever if you're wondering why i'm even doing this is because i'm going to make a video showing how many balloons would really be needed to lift the house from up subscribe for that i've been training on this deep fake for two weeks we have almost a million iterations meaning like peter has looked at a face over and over a million times you know as we dialed this in here you have to also match the colors of the film you have to match the other shots like all the things that aren't deep fake related just normal visual effects production related and then there's a whole doesn't look like luke skywalker thing we'll see if it paid off i'm just i just wanna like at this point i've been staring at it forever so yeah let's go get everybody hey guys do you want to see the mandalorian but with our version of luke skywalker yeah is it time all right guys uh we've all been working super hard on this i hope it's cool we'll see you guys tell me [Music] [Music] are you a jedi [Music] i am [Music] he doesn't want to go with you he wants your permission he is strong with the force but talent without training is nothing [Music] may the force be with you okay here we go [Music] did we achieve your dreams you know what's crazy about it is that like it looks amazing but there's like five and some shots even like ten percent dean in there blue skywalker's is ten percent deemed mark hamill if you're watching this you can have my body anytime [Laughter] the sparks look pretty freaking sweet and if you look closely you actually see the highlights and the eyeballs move too it's crazy the likeness is still not a hundred percent but it is way closer way sharper and it has way more emotion defects definitely a viable way to bring old characters back to life and if anybody's at disney's watching this like it's like a very fancy uh resume application right now we're all like kind of worn out like we've been running like full focus not just today but like this entire week all right deep fix pretty cool we've done other experiments with defects in the channel you should check out fixing the worst vfx shot of all time keanu reeves stops a robbery and the one we did with tom cruise which is no longer accurate but at the time we labeled it the best deep fake on the internet i want to know what you guys think did we do better or worse was it more vibrant or was it really weird and kind of creepy it's your guys's turn to be the vfx artist reacting in vfx rs react and we're on display here so yeah i'm going to read all those comments it's one of my favorite things to do yeah give us some breakdowns give us some critiques i want to see it subscribe

Channel: Corridor Crew

Views: 3,991,789

Rating: 4.8634238 out of 5

Keywords: The Mandalorian, star wars, luke skywalker, deepfake, cgi, VFX, vfx artists react, experiment, mark hamill, baby yoda, grogu, visual effects, tutorial

Id: 861gfPVmgdc

Length: 20min 43sec (1243 seconds)

Published: Sun Jan 24 2021

Reddit Comments

[u/akilles_xxvii]

The CGI looks wayyy better which is what they tried to prove. Ignore the acting, funny wig, and everything else that Disney already had done better because of time and resources. The lighting and realism of the face 100% better here. I’m not upset about Disney’s finished product but I felt it could have been done a lot better bc if anyone could do it, it’s freakin Disney

[u/Shakespeares_Nan]

Its obviously not perfect but I think Corridor's version looks phenomenal and far closer to a real person that Disney's. Not to unreasonably knock Disney's 'cause that I think that's still great for what it is but god damn, Corridor knocked it out of the park in my opinion. Deepfakes are moving so fast.

Corridor's method with some Disney budget hair and makeup/on location acting and a closer body double (face was very close, body proportions were a bit off) would sell the illusion to near undetectable to most people I reckon.

[u/Ske1etonJelly]

It's frustrating seeing all the youtube comments that don't understand the difference between a Disney CGI budget and a pro youtuber budget for this. This is an extremely effective demonstration at how to use deepfaking properly to get past the uncanny valley. The reason it falls short here is because they used an amateur actor, had to lip mime the lines, and didn't spend a ton of time getting the body language 100% - it is a little overacted and funky for sure. Keep in mind, their Luke had to make eye contact and emote to objects that weren't in the room with no reference for where they were. It is only meant to be a demonstration at how to get convincing facial performances, which CGI will NEVER do. If you're watching it expecting CC Luke to fit into the scene perfectly, you're looking at it for too much.

If you look at the two performances by themselves, scrutinizing only the facial movements, it looks better than the wooden CGI performance given in the Mandalorian finale. This is an example of deepfake done right, and a preview of the industry's future. Ctrl+shift+face also has some really amazing deepfake work that is extremely high res and convincing. I was screaming/laughing internally when I saw the Mandalorian finale, because as awesome of a moment it was, it was pretty jarring and weird.

If I were CC, what I would have done to improve it is have their "actor" study Luke's performances a little more to see how he moves his face during conversations. There was still a little intentional stiffness. In a situation with a professional studio and actor this wouldn't even be an issue.

[u/Ikasper23]

Aaaaand that’s how you get hired by Disney.

[u/[deleted]]

Why is the fan made Luke hella greasy?

[u/GraconBease]

I didn’t think Disney’s was too bad until I saw this. Now the OG doesn’t even look like Mark Hamill. Obviously the hair and the background is not nearly as good, but since we’re just focusing on the face, Corridor 100% takes the cake. Leagues better

[u/_JD_48]

I knew it was only a matter of time before Corridor Crew made this video.

[u/JWGhetto]

Actually better WTF

[u/TheMightyBoushh]

I still think Disney’s looks better.