Motion Capture - Retargetting

[Music] hello blenderheads in this lesson i'm going to show you where you can get motion capture data and how you can apply it to your own characters now one of the things that really bothers me about most motion capture tutorials out there is that they just go and download some motion capture data match it to their character add an armature and call it a day and this kind of works but what if you or worse your client wants you to make some tweaks to that animation let's say that you were using this basketball bouncing animation and your client says yep that's great but could you just make her stand up slightly straighter now if you were using a regular rig this would be extremely easy you just grab the hip controller and lift problem solved but with this motion capture rig it's not so easy because the legs don't have ik constraints the whole rig lifts up when we move the hips okay so what if we grab the next spine up nope that just stretches the torso what you have to do is lift the hips and then try and rotate the thigh knee and foot bones and put those back into place this is a process called counter keying it's when moving one bone forces you to move multiple other bones just to get a simple pose right and trust me it sucks because once you've spent about 15 minutes or so getting all of your bones counter keyed you suddenly realize you have to do this on every single frame needless to say this is not how animators in the industry work so today i'm going to show you the right way to do motion capture in a process called retargeting retargeting is where you take the animation from your motion capture rig and instead of applying it straight to your character you apply it to a proper rig such as the rigofy rig retargeting requires you to take each bone from your motion capture and apply it to the corresponding bone on your target rig now this can be a fairly tedious process and unfortunately it's not one that blender does by default easily so to really do this properly you're going to need a copy of auto rig pro now this is a paid add-on which admittedly kind of sucks the only free add-on i know of that does this retargeting is the rococo add-on which is designed for the rococo motion capture suit but i've had mixed results with it so far i will make sure that there is a link to the free rococo add-on in the description for anybody who wants to try and get that working but for this tutorial we're going to be using auto rig pro now full disclosure i've been using auto rig pro for over a year now and i have recently become an affiliate that means that if you use the link in the description to buy a copy i will get a small kickback for recommending it so if you're interested in getting a copy it would be really great if you could use the affiliate link as this is a great way to support this channel so with all of that out of the way where do we get some good motion capture data as many of you will already know the best place to get started with mocap is adobe miximo you will need to sign up for an adobe account but everything in this section is free you don't need to sign up for the subscription or anything and they've got stacks of motion capture data like there's currently 48 per page showing and there's at least 52 pages so that's a lot of animation it's also worth noting that they've got a bunch of character models you can play with as well this library is a little bit old so i wouldn't necessarily say these are the highest quality assets anymore but they're perfect to play around with and many of them would be fine for background characters on the paid side of things i would strongly recommend mocap online now i'm not affiliated with these guys in any way but i did buy their mobility package which is currently going for about a hundred dollars and i freaking love it it's a pack that comes with a whole heap of walking jogging and running cycles as well as things like strafing left and right turning jumping crouching all of that sort of stuff and what's really nice is that most of them link up with each other so basically if you were going to make like a third person shooter game this pack would cover 90 percent of your basic animations they also have a free demo pack for anybody that's just looking to check out the quality and my final recommendation is the rococo motion library rococo make one of the cheaper motion capture suits on the market which i am currently saving up for but in the meantime for anybody who can't afford the suit itself they have this big library of motion capture data that you can get your hands on there are some free samples in there but most of it is paid content each clip is usually somewhere between three and six dollars now admittedly these costs could add up fairly quickly but there's been times when i've needed a specific piece of animation and the only place i've been able to find it was on the rococo library so for those one-offs very specific pieces i've been more than happy to throw a few dollars at rococo and just get that animation done in a few minutes rather than you know taking days to animate it by hand so for this tutorial we're going to go free and i'm going to grab a specific animation off mixamo so i'm going to do a search for dribble and i'm going to download this basketball bouncing animation i'm choosing this animation because in the next lesson i'm going to show you how we can come in and tweak this animation to our liking the download settings you can basically keep a default the only thing that i'm going to change is to choose without skin since we're going to be using the susanne rig to retarget too speaking of the suzanne rig you can download a copy of her from the link in the description now in the past i've been selling this rig for a few dollars but for these tutorials i've decided to make her a free download when you download her you have the option of downloading both a riggify version and an auto rig pro version the auto rig pro version is much much easier to set up but i suspect a lot of you will still have rigofy rigs or maybe you're using say the rain rig from the blender foundation now these rigs can definitely still work we just have to set them up properly first so i'll show you how to set up a rigofy rig so that you know how to set up just about any rig and make it work then i'll show you how much faster we can do this by using auto rig bro so let's jump back into blender and import our motion capture data here i have the suzanne rig with the rigofy rig applied to it i also have the auto rig pro add-on installed now just remember when you first download this and try to install it there are a couple of extra steps to installing auto rig pro just make sure that you follow the instructions that come with the download so to import our motion capture data i'm going to go file import i know that this is an fbx file navigate to wherever you have your dribble.fbx saved and go and import the fbx let's just move this over to the site a little bit and you can see that it's kind of hard to tell what's going on here i'm just going to shorten this to 64 frames because i know that's how long it is and when we hit play you can sort of start to go oh okay so that's the hand bouncing you can kind of make sense of it but it's a little bit all over the place so let's actually just delete this and let's go and import that again except this time we're going to go down to the armature menu and we're going to select automatic bone rotation and now when we import this our bones are actually in a much more easy to understand location or rotation over here on auto rig pro this is how we set up a new rig so we don't want that what we want here is the remapping and here we get to choose our source and target armature so with the source that is the motion capture rig selected you can go and click this little eyedropper it'll dump that in there then we can select our rigafy rig which is our target armature and put that into there that is our suzanne rig and then we want to click auto scale which will usually scale the source armature down to something that's a little bit more like the size of our rigafy rig we then want to click the build bones list and this is going to go through it's going to go through our source armature here and grab every single bone and do its best to try and match that up to our rigafy ring now in this case it's done a pretty poor job we're going to have to set most of this up manually so let's walk through that process now i know that it's managed to get our head bone correct that's great but i know from previous experience that even though this is called the hips bone it's technically not the right bone on our rigifi rig if i click here we've got the one that's called hip and that's the one that kind of wiggles her hips back and forth but it doesn't move the rest of her body whereas if we grab the torso controller and shift that around you can see that this is the one that has our ik constraints for our legs and kind of moves everything else around a little bit more nicely this is actually the controller we want so with the hips target bone selected here we can just click the eyedropper and it will swap that over to torso whoa whoa whoa hold on a second almost forgot a very very important step here to make all of this work at least one of your controllers needs to be set as root now generally speaking in one of your rigafy or auto rig pro rigs the root controller is considered the base because it kind of moves everything now on these motion capture data rigs they don't have a base controller as such usually it's the hip so what we want to do is make sure in this case the hip or in this case the torso is set as root now setting up arms well arms and legs anything that can have both an fk and an ik constraint i will show you how to set up both but to start with let's set up the fk constraints so at the moment i've got the ik constraints selected on the arms so i'm going to need to go back to item here i'm going to need to swap it over to the fk and you see that doesn't actually change the rig here that's because we need to go down to our rig layers and turn on our fk and you can see now we get these little green controllers i'm also going to at least temporarily turn off the ik so we don't get confused and then i will make sure that this is set to fk we'll just test it there good that's working so now we can go back to auto repro go to our left arm now you can see here we've got forearm and then we've got upper arm so let's just double check on our rig here this one is called left arm and this one is called left forearm so we just want to make sure that we're matching these up correctly so arm left forearm so this is left arm which was i've already forgotten is the upper arm so we want to make sure that we are selecting the upper arm make sure that you have the left arm selected and let's eye dropper that we can then go to in this case the left foot now we've got the same problem here this is set up with inverse kinematics so let's swap both of these over and again go down to our legs and swap those around so now we have forward kinematics again let's go back here we've got left foot we actually want to make sure we've got the left foot not the left shin selected and eye dropper that so i'm just going to go through and quickly link up all of these bones and i'll be back in a minute okay so a couple of things to consider firstly you'll notice here that we've got spines and we've got a couple of them left blank that's because our suzanne rig only really has a torso control and an upper chest control whereas in our original rig has one two three four spine bones so you may need to do a little bit of guess work as to which spine needs to match up to which controller here so in our case we've got the base one which is our hips torso controller there and we've got the top spine which is spine number two on our motion capture rig which lines up to the chest controller on suzanne watch out for both spines and necks because different rigs can have different numbers of bones through them a couple of other things is suzanne he doesn't actually have a single toe control you can notice on our original rig here we've got some toe controllers there but this rig is set up to work with shoes not multiple toes so that's something that you would have to come in and hand animate uh what else is there there's toes there's spines and i think the only other thing that we need to discuss is the fingers i haven't bothered to do those yet but i'm going to show you why right now so once we've gone and set up everything that we want to set up we can export this setup so that we only ever have to do it once or at least we only have to do it once for these particular rigs to try and match so if we come down here we can go under mapping presets and make sure that we've got this rig selected we can export this now i've already gone and done this before you can see here i've got a remapped miximo to rigofy rig i did this yesterday and i've actually got my fingers and everything set up in in that preset so rather than wasting the time here i'm actually going to go import now you can see here on our little arrow we've got a whole bunch of presets here these work very very well but they only work with the auto rig pro rig i'll show you that in a minute so let's go and import and let's go and do the one that i created yesterday let's import that mapping and you can see now all of my fingers have been filled out this is great it means that we really only have to do this once so one last thing that we need to do before we go and retarget we need to redefine the rest pose on our motion capture data here so if i just go back to let's just scroll to frame 0 i will go to pose mode i will select all of the bones and i will reset its position and rotation by holding down alt and g and you can see that in its default position this rig has a t-pose whereas in suzanne has an a pose this will mess up things later so what we need to do is get this rig roughly in line with suzanne now fortunately we don't have to do this manually the uh auto rig pro comes with these settings for us so we do need to make sure that we are in object mode and then click redefine rest pose this will give us the option of using either a rest pose which we can define or a current pose we don't want to be using the current pose because again this is in a t susanna's in an a so keep this on rest pose and click ok now we want to go in and select the bones that we want to rotate now we can just select all and just go copy i found that that can mess things up a little bit so i know which bones here are most likely to cause us issues we can definitely see that it's the arms so let's make sure that we go through and select all of the arms and i think her legs are just a little bit more splayed than this one so i'm just going to select the legs in this case i'm not going to select the feet i've actually done some experiments and in this case it does mess these up so just the arms and the legs click copy you can see that that now pops into place if for some reason some of your bones aren't targeted properly you may end up with weird results make sure that you go back and change the bones and then we just click apply and we can go back into object mode and this is now all set up so with everything done let's click retarget and you can see that this can automatically detect how many frames are on your motion capture data that's a nice little feature click ok give it a moment depending on how how many frames you have and then we can play the animation and you can see that this has definitely worked now this rig is set up so the eye controls don't actually follow the head so let's just shift those around to the front again so she doesn't quite look like zombie suzanne and let's admire some of our work so this is looking pretty good this is very very close to being usable in final animation but i'm noticing it's not too bad on the hands but if we look closely at the feet you can see that they're wobbling ever so slightly and that's just a little bit of noise that's built into our motion capture here so we can either go in and we can manually clean that up ourselves on all of these different bones which is a bit of a long and tedious process or alternatively at the moment we're using fk forward kinematics on her legs here what we can do is swap these over to ik which then makes it a little bit easier to plant her feet where we want them so let's just go in and let's retarget some of these bones and i'll show you how to set up ik so firstly let's go back to the suzanne rig and let's make sure that these are set to use ik that's back under item we'll just swap that you can see that it pops back into the default place for the ik oops i want the torso on turn that off and let's make sure that the ik arms are set up we can turn off the fk for now and then we will do the same with the feet select that swap those over and turn off our fk and turn back on our ik back here in auto rig pro we just want to find our arms sorry our hands and our feet so we've got our left foot here let's make sure that we select the left foot and swap that over to ik now to make this work you also need to make sure that you select ik down here and you also want to grab your pole vector constraint this is the one that allows the knee to wobble back and forth and make sure that you have the pole set and then you can do the same for the hands ik make sure that we've got the right controller there and set that as the pole and scroll through here until we find the right side not the arm we want the foot select that make sure that it is set to ik grab your ik constraint now sorry your pole vector constraint then grab the right hand set it as ik and set up the pole vector so now with that set up firstly it's probably worth going and exporting this again so in this case i will call it miximo rigify let's go underscore ik so we now have an ik and an fk if we should ever need both we will re-export that and let's go and tick retarget now when we play this animation first of all you can see that these feet don't seem to wobble quite as much that's kind of nice but what's really good is once we go into pose mode here we have full control over these keyframes so i'm going to go and add in a ground plane just scale this up a little bit and let's get her feet where we want them to be so because we want our feet to stay planted we can actually select both of these controllers now we can actually delete all of those keyframes we don't actually want them to move at all and then we can go over to item and we just want to make sure that this is set to zero on the z-axis so that it's not penetrating through the floor you can see there's a little bit here but that's because we now need to fix the rotation so the z rotation is what gives it its uh around rotation so we want that to stay at that 70. let's set both of these to zero you can see that tilts her foot back and then we can do the same thing over here set those rotations to zero set the location to zero and now you can see that she's standing on the ground quite nicely and if we go and play through our animation you can see that now the feet don't wobble and she's bouncing that imaginary ball very very nicely okay so we have another scene file here this is the auto rig pro rig and you can see that although it is very very similar this is a slightly different rig so i'm now going to show you just how fast this is to set up if you've got auto read pro installed and you're using an auto rig pro rig so again let's just go and import let's import our dribble remembering to check the automatic bone orientation import that let's just shove him over to the side slightly we will select him as a source armature we will select the suzanne rig as the target and quickly auto scale that just to get our sizes right and now you will see that when i hit the build bone list that because auto repro knows its own rig it can go through and fill this out automatically this obviously saves us heaps of time now i have noticed that um for whatever reason there are a couple of little mistakes in here so for example here you can see that the right leg up for some reason has been linked to the left thigh so you will probably want to change that and from memory there are a couple of other little like really little mistakes in here but because we're using auto rig pro we can actually go to our mapping presets we can go to our little arrow here where we've got some defaults and we can choose because we know that this has come from miximo we can choose one of the miximo presets so we experimented with the fk before and we found that the foot was slipping a little bit that was something we wanted to fix so this time let's choose the miximo ik and you can see that it changes a couple of little things in here it's uh removed the spine it it's it's swapped over some of our fingers um sorry that was something i should have mentioned before suzanne you can see only has three fingers and a thumb whereas in this is based on a regular human and it's got four fingers and one thumb so this is smart enough to be able to go okay i'm going to remove one of the fingers from each hand and we'll just double check that the hips has the set as root which is working we will still need to go in and redefine our rest pose so let's do that now select oops just select don't move them select our arm bones select our leg bones making sure not to select the feet because i know that those can mess up let's go copy the rotation and apply it let's set this to 64 frames again and let's go retarget and you can see let's just turn our overlays off we have immediately got our animation set up and again you can see that our feet are moving ever so slightly so i might just go in here and do the same that we did before let's just delete those keyframes go to the item just make sure that these are set to zero on the axis we know we want them on i'll again just throw in a plane so we can see her on the ground and you can see boom in like i don't know a minute or so we've copied our motion capture data across and if we want to we can always come in here and uh move things around now in terms of being able to change some of this animation we have the same problem that we had before if i just turn auto key on and i shift her up so that she's standing up ever so slightly you'll notice that i go to the next frame and she pops back down so this keyframing here because every single frame is keyframed that's doesn't really work very well for us fortunately auto rig pro has some tools built in to help us deal with this let's go down to the interactive tweaks uh you need to make sure that you have whichever controller you want to move selected so here you can see that we do have the hips make sure that is selected and we can go to either our rotation or location and we can just tweak this a little bit so i think i want to move this what up in the z no that's actually moving her forward so let's let's go the y then sorry rotation plus y and you can see now we can start to move her up and down a little bit so you can see here we're getting a little bit of pop on the arm so let's select her hand we want the which one's this this is the left hand so let's make sure that we've got the left hand selected and we might also just pop that up a little bit and you can see now that she's standing ever so slightly higher and the motion capture uh stays on the rig it doesn't break so this is one way of being able to come through and tweak your animation after the fact in the next lesson i'm going to introduce you to the animations layer add-on which gives us a whole heap of more control so that we come in and much more easily make some of these tweaks so i will see you in that lesson you

Channel: Jamie Dunbar

Views: 16,218

Rating: 4.9312716 out of 5

Keywords: Blender, tutorial, Blender 2.9, motion capture, mocap, auto rig pro, animation layers, retarget, retargetting

Id: AutNCxcK0Xc

Length: 25min 6sec (1506 seconds)

Published: Tue May 04 2021