Animating with Motion Capture for the First Time

hey everyone welcome back i'm suede and i've never used motion capture before today we do it for the first time and i'm going to use it to make three different things so attempt number one was primarily just using the motion tracking gloves attempt number two i did the full suit and actually figured out an animation workflow to work with and customize the data send it down the pipeline and do some stuff with it and for the third and final attempt of the day i took everything i learned i took the suit brought my laptop i went to a parkour gym here in la and i filmed out in the field got a bunch of motion capture data that i brought back and tried to make something as quickly as possible so that's what we're up to i had a ton of fun with this i learned a ton and i failed even more i'm going to share all that with you now so here we go and last thing if you enjoyed this video please hit the thumbs up and subscribe down below it's free so why not you can always change your mind later enjoy the video let's get into it today i have these which if you've never seen this before these are the rococo smart gloves which is a motion capture glove situation uh they're plugged into little battery packs my data is being tracked and placed on the character which is very cool watch this watch this that's not good we're gonna make something cool with the laptop today if you haven't seen my last video on this computer nvidia sent me a laptop with a geforce rtx 3080ti and that's what we're doing stuff with the entire idea came from knowing that these existed and seeing these i was in a camera store i saw them on the shelf and i immediately knew exactly what i wanted to do i wanted to do something cool with my hand key out that green suit and have just like kind of a stump of an arm and combine that with this i figured if i had my hands being tracked i'd be able to have a 3d replacement for the hand that i'm keying out which actually turns out to work pretty well so the first thing i did was just try the gloves i didn't bother with the full suit i thought maybe just the glove would be enough so that way i'd be able to key out everything i did in the actual camera view and simultaneously record all the motion in the computer in the laptop the good thing is i got all kinds of cool finger animation really really quickly huge while i didn't really need the full suit to get all the hand stuff i was actually moving around and stuff and it wasn't capturing like my shoulder and my body position it was just doing kind of an ik hand situation so while it kind of worked for where it was it wasn't actually properly tracked to my physical location so i had to manually animate sort of the shoulder joint to follow the footage i ended up deleting and redoing all the animation for the upper arm and the elbow i realized that i could have bypassed all of that if i had actually used the full suit the real problem and this is the thing that i i hope someone in the comments can tell me what happened here or what i did wrong i don't know if it was me or if it was maya but i captured all the data inside rococo it looked great there i exported it to an fbx file which is normal and then i imported that fbx file into maya directly and started adjusting the animation so i left that file open and worked on it for six and a half seven days straight you know modeling the hand to look like a t800 terminator hand like it looked really cool what you're seeing here is a random geometry cache export that i happened to do like a week before i did a lot of that work i just mentioned eventually something happened and i had to close my restart my computer or something when i reopened the project it was empty all the joints were like sitting at origin there was no curved data the mesh was gone like everything i had worked on had disappeared and i to this like i don't know what happened that sucked that was like a week's worth of work gone lucky for me i at least had some of the cleaned up data exported with kind of the default mesh so i threw a new shader on it used motion blur and i rendered this out i didn't bother masking it and making it look proper but you can see where the direction was going you can see where things were headed and it's pretty freaking cool right unfortunately you'll never see how it actually was meant to look i'll maybe do that in a future thing when i am emotionally ready to tackle that again but no use crying over it let's go to attempt number two so i decided to use the full suit this time so i came up with two different scenarios i figured i'll do something with dialogue in case i want to do a little acting thing and i'll do something more body mechanics in case i want to go that route my intention was to replace one of my selves with motion capture so i did both characters in regular clothes and i did both characters then again in motion capture that way i'd have all the footage all the data i suppose you think that was terribly clever in my head i was doing a little count like on the three second mark punch on the four second mark second punch that way for both or all of the characters that you may be doing stuff with you know that you're hitting those beats at the right moment i just realized in real life i was having to stand in place and back up into a wall but here i could stand right here i can stand here and go and like jump backwards because i can just reposition the character i'm doing too much footwork i think if you get kicked you're not stumbling back with many steps if you get kicked you're getting launched it was a real life situation where i just kind of stumbled backwards and it didn't really feel like i got kicked i sort of set my outer bound so i didn't mess anything up but i had it so that when i got kicked i just sent the translation data a lot further back and i messed with the interpolation so it felt like the kick happened and bam character goes and flies a lot farther than they did before and then all the animations just sort of offset from that point and then i also messed with the legs and the arms and stuff to make it feel a little bit less like i'm stepping and more like the legs kind of get kicked off the ground and then come back and and catch hopefully you can see the motion and you can see that on one hand i've got little flat hands that don't do anything and on the other side i've got hands that actually react to my finger motions which is really cool and i've got spatial tracking to some extent the gloves in particular have one massive sensor here the hub that kind of controls everything we've got one two three four and five sensors and the fingers and there is one on the base of the wrist but uh this motion is basically just copy pasted from the joint above it and then reduced a little bit we can adjust all of that in whatever software and there are a few ways that we can go about doing it here you can see that one hand has some data if i let it play it actually has animation on it the other one does nothing just a flat panel so i grabbed every finger control and to make life easier if you're not using animbot you can just open the script editor copy all the code for grabbing that stuff and make a shelf button so that you can easily select and deselect all those joints at once under animation layers i create a layer from selected this reveals the base animation layer which contains all the animation data for the rig but now i have a specific layer just for the finger controls in that layer i can repose these controls to be whatever i want by doing it this way not only can you just animate it normally without influence of any other controls that may previously exist but you also have a way to turn on and off the effect with the weight slider on this layer and you can key this influence so if you only want this to happen for a little bit and then go back to whatever the default was you have that control as well at first i realized that i was just doing things really ineffectively inefficiently just bad i was like picking bones in the the human ik editor and like hips hips spine spine i was just doing it the slow way i realized later you can just sort of click on a skeleton and load the preset stuff and apply it like it's much faster if the data let's just go full 3d for both characters these little bass mannequins are working just fine and i'll just do a fully cg environment i won't even worry about this whole live action integration thing we'll save that for a future video because i will conquer that i tried to export that data to an fbx which is how it came in i got this monstrosity i don't know what went down there so instead i just brute forced it i normally do an alembic cache which is much heavier and much slower but i decided to try usd because i do some stuff with usd lately and i'm trying to get more into it now i could have rendered all this in arnold but i'm actually really enjoying the shader workflow in blender so i exported the data using usd into blender and then started working on the environment i used some assets to kind of populate the set and then for shaders i just started exploring if you're working trying to work quickly ev's usually all you can do because it's it's pretty real time cycles takes a little bit longer unless you're using a good graphics card so let's briefly talk about what matters in the computer for a project like this obviously having a fast processor helps us to animate maya and having a lot of ram helps us to just have lots of data in the scene in 3d but the real star of the show today is going to be the gpu having a fast graphics card in this case the geforce rtx 3080 ti is going to allow us to preview everything that we're doing in real time a lot of times in animation we don't really worry about lighting and rendering because that's not really our job that's going to happen after our stuff is done depending on the studio environment that you work in you're going to be taking your data and pushing it down the pipeline for someone else to worry about but when you're making stuff for yourself it ends up just being your responsibility because you make everything so instead of waiting until the last second to kind of do that after the fact i wanted to see if i could actually include that thought process up front i could actually hit play and just watch the animation with cycles kind of rendering it so i got a sense of what it was gonna look like from the beginning not just the one frame but the entire thing i also did camera animation and i used uh the blender modifiers of the graph editor to create little like noise spikes that i blended in and out so every time there was a good impact like the kick i had a little camera shake that i didn't have to animate manually i just used the noise to create what i was looking for i wish maya had that i really liked that feature but building the shaders and being able to do it in real time using cycles where i was getting proper reflections all the stuff was accurate i was getting the ray trace lighting it was working really really nicely and i wasn't waiting for anything it was all immediate feedback super fast loved it so i got to try out some different stuff with lighting because of that i got to try some more kind of pink hue i tried sort of a blue cloud thing i decided to go for more of like a sunset look just because i liked the way it caught the buildings all decisions that i would not have been able to make had i not been able to preview my animation with the lighting in real time so that was really nice i also got to check shadows so at one point i don't know if i have the footage for this but at one point the character moved their hand in front of their face and it caused a pretty significant shadow on the face i would absolutely have missed that if i was not able to preview my animation like render quality that quickly and then i would have rendered spent all that time to render an animation that had a big old shadow and i would have to redo it so legitimately being able to work that quickly and use something like the 380ti to iterate and to see the progress as i'm working on it that was a huge thing and same with reflections if you zoom in you can actually see the reflections of the other character because i was using a metallic shader so you actually get to see all the reflections and things like that but ultimately i was able to make this scene pretty easily all things considered now on to the third and final attempt number three so i wanted to bring everything that i had learned so far all together and just make something as quickly as possible really just within a couple of hours so i brought my laptop the mocap suit all that stuff over to the tempest free running academy the parkour gym that i've been going for about 11 years it's been a while so i didn't want to do anything too crazy my biggest bottleneck is that i didn't update my suit before i went so i actually had about two and a half hours of troubleshooting as i tried to get everything working which was really just my bad pro tip just make sure you have all your stuff updated before you go try to shoot out in the field so learn that the hard way eventually i got everything working ran around did some dive rolls did a few just smaller acrobatic things just to get some data one thing that's really interesting is in the rococo studio app there is a series of filters that can help you with your mocap cleanup different locomotion things to help lock the feet down for example those normally work great but when you're doing flips and rolls it's trying to have your feet locked to the ground as if you're walking which breaks a lot of these types of action so i turned a lot of that stuff off and ended up with something a lot more useful now something was going on with the foot i'm really not sure why but my feet were just inverted every time i put my toes down the feet would tilt up and vice versa but anyways i took a section that i liked i exported it and instead of trying to clean it up i decided let's just throw it straight into the computer i also set it to the treadmill mode so that the character doesn't actually move through space i was going to do that manually so i just sort of looked for the footfalls and manually placed the character keyframing them to move through an empty scene it was interesting because i could tell exactly where the blocks were supposed to be and so i just built the environment around the motion i didn't line things up exactly and so i ended up having to kind of move things at different times to try and make it feel like the character was actually hitting the right spots but that gave me a great idea i decided to do sort of a moving obstacle course something that i've never really seen before in a parkour shot i just started having everything move around and fly around the scene so i just went with opposing colors and started playing around i knew i wanted to have some kind of a reflection and so i decided to sort of warble that reflection so it's not so perfect i also wanted to glow and i figured out how to make it have these kind of digital vertical stripes of glow and that's pretty much what i went into i went for this sort of tron sort of look but in total this whole process probably took somewhere between two and four hours if you include all the motion capture stuff at the parkour gym and all of the actual 3d work that i did here super fast to come up with something and have a bunch of different ideas to try and for only my third time doing anything i'm pretty happy with the amount of time i feel like it's just going to keep getting faster as i get better and more efficient at this process but as a first time working with motion capture in general i got to try a bunch of different ideas in the span of like two or three weeks and what was really cool is that usually as an animator it takes a long time to animate that stuff i was able to do a bunch of different things the amount of hats i got to wear and and little pieces of the projects i got to test animation was not any kind of a bottleneck for me animation was just a short sort of it was a clean up and a pushing and you know there was a fun artistic component to the animation to take the raw data and adjust it and make it more interesting and make the poses stronger and i could have gone a lot farther than i did but just in the short amount of time i got to use motion capture as a tool to speed up that part of the process still have a lot of creative control over it but then try bigger things and that was the whole point i'm really happy with how some of this stuff turned out just from the sense of like the possibility of what these things can be i'm really excited to go deeper into this if you have any ideas requests anything that you want to see or learn or whatever let me know in the comments like you know keep me up to date with what you're looking for because i want my learning experience through motion capture and the game's workflow to benefit you as well that's kind of the whole point of this youtube channel as it always has been is to document my creative journey so that you can hopefully learn from my mistakes and successes and we can just grow together that's the whole point so a huge thank you to nvidia studio for being a part of that learning journey and helping to provide technology that allows me to work quickly and make cool stuff and not have to sit there waiting for a render instead of doing cool things thanks nvidia studio so if you enjoyed that and you want to see that kind of stuff hit like it actually helps these videos a lot and if you don't miss the new stuff be sure to hit the subscribe button down below and ring the little notification bell so you don't miss out on when new things come up and i'll link to all the other stuff down below patreon if you want to join the behind the scenes or my animation mentoring stuff hope you enjoyed this video i had a lot of fun i can't wait to do something cool in the future and uh thanks for watching i'll see in the next one [Music] well [Music] you

Channel: Sir Wade Neistadt

Views: 88,486

Keywords: motion capture animation, animation mocap, working with mocap, mocap tutorial, blender mocap, maya mocap, motion capture tutorial, mocap suit, rokoko, rokoko blender, rokoko maya, animating mocap, how to animate mocap data, first time mocap, maya first time, blender first time, exploring mocap, first time animation, mocap, motion capture, xsens, mocap jobs, learn mocap, animation layers, mocap gloves, hand animation, mocap hands, vfx animation, parkour animation, 3d, b3d

Id: WXJtxEirtW0

Length: 15min 27sec (927 seconds)

Published: Mon Mar 21 2022