Creating a Mocap Fight Cinematic in Unreal Engine 5

thanks to Dell and Nvidia for sponsoring this video Unreal Engine 5 is one of the coolest 3D tools out there and I've been doing a ton of stuff with it especially animation a bunch of videos and workshops coming soon by the way but today I want to try out a motion capture workflow to create a cinematic sequence as always we start with the planning since I'm doing motion capture I don't want to do anything too styliz I want to take advantage of the naturalistic motion and do something hyper realistic I also have a ton of assets through Quicks so Bridge the unal marketplace and cargo the subscription I have where I can just pull a bunch of different assets into my scene and since I currently have access to the xn MVM link suit some pretty Advanced motion capture Hardware I want to do something where I act out multiple characters and create some fight choreography and since that's already enough of a challenge on its own let's not worry about facial animation facial capture let's stick with something faceless but humanoid so most likely robots I seem to really like doing robot animations so I looked at reference like Edge of Tomorrow Terminator RoboCop District 9 code 8 anything that had that liveaction hyper realistic humanoid but robotic type of character so with that reference in mind I came up with a rough idea of a story something with an underground bunker some secret military prototyping A Deadly Assassin killer robot type thing not super original but whatever it's fine I'm going to start off with cargo from kit bash and just start populating my scene with a bunch of assets the reason I'm shutting with cargo is they have some kind of like microphone situation around my office because this is the second time in a row that I've had a project that I'm like man I wish I had assets that looked like this and that week they come out with assets that are exactly what I'm looking for I don't know how they do it but I'm going to take it also affiliate link to Cargo down below but I'm going to start building my scene using my references as inspiration after a while it started to feel like an arena and I needed it to feel underground some kind of Bunker type Vibe so to make it darker let's work on the walls let's work on the ceiling and I need something that looks like the scene in Edge of Tomorrow I really like this whole Square ceiling thing so I jumped over to Maya to create that because I'm most comfortable modeling in Maya so I built the little square bevel thing used a mash Network to create an array and connected my session to Nvidia omn which allows me to position and texture it in Omniverse and omal engine any changes I make in May will automatically populate into the other software I don't have to export back and forth I can just see the updates live in real time with all the lighting and make sure I like it and I can do this with botels textures lighting camera all kinds of different things this is one of the big reasons why USD workflows and Nvidia Omniverse are such powerful collaborative tools as both individuals and for Creative teams and I've covered Omniverse before the USD based 3D creation and collaboration ecosystem it's free to download and I have a whole video on Pixar's open USD and how it's used in production here but anytime I'm making custom assets I'll typically just do this make my stuff in Maya live link it over to the other software and then just texture it with the content browsers and the libraries of materials that I have in the other tools speeds things up quite a bit and I can make changes really quickly and with the environment made it's time to move on to previs I did some rough thumbnailing at the beginning to get a general idea of what the shots might be so I'm going to use that to block it out I'll just drop in some unreal 5 mannequins and I'm going to use animation sequences to get some motion without having to actually animate them myself or put on the suit just yet so I'll download a free pack of General motion capture animations and I'll pull out some death animation sequences and just put those onto these characters bake it down to the poseable control rigs delete all but one key and just adjust that pose for each character I can also bake that to an animation sequence so that it can work across all my shots now for the main characters I'm going to apply a bunch of different animation sequences and just blend them together to approximate the direction of what I want them to do so I'll throw in an idle a transition to standing some aiming so that they're holding something because I know I want them to have some kind of weapon and ultimately I want it to be a sword because I have a foam sword prop that I can use and I specifically want to try out a motion capture prop tracking workflow but because these particular Clips have the character in a pose as if they're holding some kind of rifle I'm just going to go with that for now and I'll change it later so I brought in this laser Blaster and constrained it to the character and if the prospect of doing control rig animation natively and I'm real and constraining things to your characters stuff like that appeals to you make sure you subscribe to for upcoming videos and keep an eye on the course page link down below because there's a lot you're going to want to keep an eye on so with a character moving around we can rough out different camera angles to block out our sequence I'm going to play with focal lengths angles framing I'm going to dial in the settings at this point for my camera to pick my aspect ratio the look of lens flares in camera VFX color things like that just so that I like the tone of what I'm seeing now here I wanted the character to stand up from the last shot look around and ultimately discard their weapon now again I don't want it to be a blaster or a rifle or something I want it to be a sword and this was double confirmed when I looked at this scene and thought you know this is pretty gruesome having the character holding some kind of rifle looking thing and there's like characters that are humanoid just draped over objects in the background not a huge fan of that I'll definitely be changing that but the map sequences that I'm using aren't giving me the performance that I need so I'll throw on an additive layer track of animation with control rig and animate on top of the motion capture have the character look around lead with the head and ultimately move the arms a bit unconstrained weapon and have it fall to the ground at this point it's time to watch it and see how it plays unfortunately it's a bit boring I was going for sort of a cinematic build and instead it just felt slow it left all the action until the end which I haven't even done yet my thought was to have this character fresh from a fight looking around discard the weapon walk towards the door only for it to open and a new wave of enemies come in for them to then effortlessly deal with obviously I haven't gotten that far and I've already created enough challenges I figured that maybe we'll just focus on the fight itself that left all these robots decommissioned in the first place that way I can just jump into the map suit now off camera this whole time I've been moving the entire process of what you've seen was done at my parents house between LA and New York which is where I was living and where I'm moving to and I was doing it all on a laptop specifically the Dell Precision 7780 mobile workstation with all the right upgrades for this type of work and also twice as much RAM as my normal Tower PC which we love it's an Enterprise level machine built specifically to handle these types of professional workloads with industry- level performance reliability support and security as you saw every part of this workflow has been done with several different 3D applications running simultaneously with real-time lighting enabled and everything being captured in 4k on this workstation it can do that cuz it's powered by an Nvidia RTX 5000 and if you're hitting a performance ceiling on other laptops workstations like this are the next level up these types of machines are built to be deployed into Studios and large companies with you know hundreds or thousands of employees that everybody needs to be working very very quickly with a lot of power and now you understand why this video was sponsored by Dell and Nvidia they also shared with me a monitor and keyboard mouse combo so that I could use it as a desktop computer just run off the laptop which is what I did probably most of the time so we moved into our New York apartment and I put on the suit xend let me borrow their mvn link suit which is pretty much the fanciest map suit you can get that has its own built-in sensors inside I'm also using the Manis metag gloves to capture hand animation which also I'm going to be doing a video testing the Manis metag gloves as well as the stretch sense Fidelity and Studio gloves and real Coco smart gloves but all of that for another video the link suit comes with an optional prop tracking sensor which I'm going to use to track any offset in the swords animation from my hand this is going to save hours of time so we'll just velcro that to the sword and we're good to go this will prevent the sword from just mirroring exactly what my hand is doing it'll be able to track its own rotation so if you spin the sword or have any just general finger movement that manipulates where the sword is you get accurate prop data that's very very important with the sudon calibration is just an idle pose and walking around a bit for the body doing some certain actions with the hands and we're good to move on to acting and choreography I felt at this point it was important to decide which characters I'd be portraying so I found these models on the unil marketpl and I bought them they're compatible with what I want to do but they're not ideal and I'll show you why later on and since I was going into the fight section blind I just started trying out moves that I thought could be cool paying special attention to exaggerate the weight and the timing of my actions so I'd get a head start on the character animation modifications that I knew I was going to need to make so that it didn't feel like I was just flailing around with a weightless foam sword wanted it to feel like there was heft in this weapon and that there was resistance when slashing through metal or pulling it free of one of the other robots I also tried to keep the character's physicality of mine so that I actually felt like a Droid moving around and not a human in a super expensive onesie now what doesn't help is that I have no experience with swords so my technique is probably abysmal so it's a good thing I'm not doing some historically accurate medieval piece that's not going to work but robots whatever what I really had fun though was playing the role of each robot in order trying to imagine what each other character was going to be doing and when they were doing those actions so where I look the actions I take and the physical impacts all roughly line up without me having to rtime and redo all the animation and change all the distances between the characters this is what it ended up looking like and I think it got pretty close the step after this is processing editing and animation the xn MVM software has a processing feature that will automatically clean up Jitter foot sliding and any weird irregularities in the motion there's not much I need to do in terms of General cleanup and it'll even keep track track of the world space relative to the actions you take so if you jump on a treadmill in place it will calculate where the character needs to be to make sure that the Motions you were doing makes sense I was super impressed with the quality to get it to Unreal Engine we can either stream the data live from mvn over to Unreal or we can just export it into a normal file and bring it in that way to do that you need to download the live link plugin to get unreal to read the streaming data and you need to get the skeletal mesh asset that has the joint configuration for this character that's compatible with the motion but if we throw it all in unreal we going to see that the actions actually line up pretty well almost well enough to just leave them alone and there's a number of ways to fix the timing I'm going to go ahead and just have those exported fbx files brought over to Maya for the retiming I tried a few different things I tried manually retiming the motion in the graph editor and the dope sheet which is what I would normally do for your traditional key frame animation workflow but with the density of key Frame data here and how long these takes were this was a bit hard to do with so much information so for the first time ever I tried out the time editor which was designed for this very purpose so I imported all four clips of Animation I created a speed curve for each one and I key framed the speed of time for these animations to sync up all the different actions I retimed the sword swings to make them faster and more powerful as well as making the blocking snappier so that ultimately everything just felt more forceful and powerful and it all lined up and there's still a lot of other animation adjustments I'd like to make and this would be a good step and a good place to do those changes especially because the human ik editor gives us these little rigs on the character that we can do FK we can do ik and we can make any adjustments we need to make pretty easily but why make the process so easy let's do it the hard way so skipping ahead to retargeting an unreal retargeting is about having the motion meant for the xn skeleton asset to instead manipulate whatever other character we want to apply that data too and I'd like to point out that up until this point the animation capture quality has looked great and anything that makes the animation look bad on my robot from this point on is because I wasn't super careful when I made my ik rig and retargeting assets so this is going to introduce some Jank that you could always come back and correct later but I didn't so we basically just tell unreal hey like this is the upper arm joint this is the lower arm and so on so it understands where to send each part of the motion data from one character to another then we take each retimed animation sequence from Maya and we run it through a retargeting asset to create animation sequence clips that are looking specifically for the new joint definitions that exist in the robot characters that I downloaded then with them moving around we bake that animation to their FK skeletons so I can modify the animation from here and you remember how I said that this was the hard way this is why when we were in Maya we had different controls to modify the exence data using FK ik all the different stuff we'd use as a character animators these robot assets and unreal do not but it's not the fault of unreal it absolutely supports it if you do this process with an asset that has a control rig this process isn't nearly as bad it actually is totally fine you can just bake it to the control rig and do it like you would have done it in Mya that's why I said that these assets in particular aren't super ideal for what I'm doing specifically because I'm an animator and having assets with a control rig built in would make this process much more enjoyable without any extra work and since I'm not going to make a control rig for these assets right now I don't reap those benefits and ultimately all I want to do is reduce how much wiggle the swords have so they don't feel like there's some foam prop shaking around I wanted to feel like there's force and impact holding them in place I also adjusted the position of those swords when they Collide so they actually come into contact but I'm not going to spend too much time worrying about the floor contacts and the floating and stuff since most of what's actually broken here is because of my shotty retargeting rather than the animation data itself like I said if I went back and fixed the ik retargeting asset those broken things would actually fix themselves but instead moving on now I had one of these weapons to be interesting bladed weapons or some kind of club or just you know something cool but at this point it didn't seem that important so I just went with an emissive material and stuck with the kind of lights sabery look oh well but I wanted something every time the beam swords hit so I added Niagra systems to emit Sparks of orange when the robot's molten metal was swiped out from them or bursts of blue for kinetic energy and with all that triggered in the sequencer it's time to redo the cameras now I'm not the greatest cinematographer but I was working on a side project with a friend of mine who actually does some really great camera animation he agreed to take a stab at the cameras for me and I ended up with fbx camera animation that once again through aidia Omniverse syncs up between all of my applications these are the six different camera angles he gave me and while we could have done specific Cuts within the engine to only render what's needed just like in the previs how it just did the one sequence I treated it like a live action shoot rendered each camera out for the full sequence and just treated each shot as coverage so that I could edit it in any order in post it took a lot longer to render that way but again they Precision 7780 workstation and an RTX 5000 GPU it was dumb when I woke up the next morning for the final touches I'd already kind of handled the color in the camera settings and so it was time to do sound design for the edit and the last things I'll say before you see it first there's definitely some Jank I blame my shotty retargeting and lack of proper animation cleanup if I had left these as the xen's little skeleton assets they'd actually be a lot more stable they just wouldn't be cool looking robot with interesting textures and secondly I think I might actually do a part two to this add some metahumans get some human facial animation motion capture if that's a workflow you want to see or you want me to make that part two of the video let me know in the comments but with all that finished here's what it looks like so far anomaly detected [Music] offline thanks again to D and Nvidia for sponsoring thank you for watching and as always I'm s Wade I'll see you in the next video

Channel: Sir Wade Neistadt

Views: 11,901

Keywords: unreal engine 5, unreal animation, unreal 5 animation, ue5, ue5 animation, ue5 cinematic, unreal cinematic, unreal 3d, ue5 mocap, unreal mocap, mocap workflow, Xsens mocap, Xsens link, mocap tutorial, mocap animation, mocap cleanup, animation workflow, animation, 3d animation, animation process, unreal 5 animation tutorial, animate in unreal, unreal previs, motion capture animation, unreal tutorial, unreal engine tutorial, ue5 tutorial, learn unreal, unreal jobs, jobs, ue

Id: r8HK4QFSQpw

Length: 14min 43sec (883 seconds)

Published: Sun May 05 2024