Inverse Kinematics in VR: How to Use Unity to Control Your Arms and Hands

Video Statistics and Information

Video
Captions Word Cloud
Reddit Comments
Captions
Having floating hands really freaks some  people out. Now there are many games that   get away with it pretty well. But if you're  like me, I'm the kind of person who likes   to be as realistic as possible. So let's add some arms and a body to   our floating hands. All right, just like last  video, I'm going to copy and paste the realistic   physics hands tutorial project and rename that to  VR ik upper body and then open up unity hub, Unity   just released a 2020 point 3.1 Long Term Support  version. So make sure you go and upgrade to   that long term support means it's stable. And  they're going to support this obviously for   a while. So this is a good version to update  to All right, now that this is installed,   I'm going to go to projects and add that project  that we copied and pasted select folder, and then   we're going to upgrade it to the new version and  confirm and there's a few things that we need to   do to just upgrade to the newest version. So go  into Edit Project Settings, go to the XR plugin   Management and click on the exclamation point  fix all to lock the input that'll fix that.   And then let's go to Window package manager and  then make sure you're in Unity registry. And then   we're going to scroll down. And anything that  has this little up arrow next to it needs to be   updated and XR Interaction Toolkit is actually to  a 1.0 version. So that'll be out of preview very   soon. It's technically at a preview now, but it  still has that pre dot three tag at the moment,   once you update all of those, we should  be good and set up for the new version.   So let's go set up our scene go into the sample  scene that we've set up from the previous project,   you're going to have that grabbable cube and table  and I'm just going to delete both of these and the   gizmos, I'm going to take the 3d icons down to  very small this player that we set up last time,   we're actually not going to use so I'm going to  uncheck it to disable it. But we are going to   kind of replicate how the components are inside  of here. So that's why we're leaving it in the   project. Instead of just removing it completely,  we're going to come back and use it as a   reference. And then let's bring our new character  in. So I'm gonna go to package manager and my   assets. And I've already downloaded a speedball  character asset, it's free from the Asset Store,   I'll put a link down in the description. So we're  going to import this guy. And of course, any   humanoid character will work if you're following  along with me just make sure they're, you know,   human light. So we're gonna go in here, grab that,  drag it in, reset it to the center, and then I'm   just gonna rename him to character. If we look  inside of this package, you'll see we have a body,   which is just the mesh the visual components, so  if we unchecked that you can't see anything. And   then we have the this BIP 001 game object. And  if we hold down alt and press the arrow, it'll   open up all the trees underneath. And we'll see  that these are all the bones for this character.   And we're going to be working with the hands the  head and then eventually the feet when we get to   that point, but trying to select the right bone  from the tree and having to like go down and open   each tree is kind of annoying and difficult. So  we're going to import an animation rigging package   from the package manager, I'm gonna go to unity  registry and then look for animation rigging.   And this just recently got a 1.0 version as well.  Unity is finally getting to a place where things   are more stable. And it's really exciting for me.  Once that's installed, you'll see this tab up here   at the top, and it gives us a few options. So  if we select the character, and then go up to   animation rigging and click bone Render Setup, now  we have this like bones skeleton that we can see   from the outside of the character. So if we wanted  to select the forearm, now, we can just click on   it, and it selects the correct bone for us. And  we don't have to navigate through this whole   setup here, which is nice. And from here we can  you know, like you normally would, you can move   objects and reposition the character's hands and  whatnot. This is like how you would do animations   and all that kind of stuff. You would keyframe  each one and do animations and whatnot. But we   want to do inverse kinematics. And then we want  to match the target of the inverse kinematics up   to the controllers, then that way, anywhere that  controller goes, that's where we want the hand to   go. And wherever the hand is, that's like you know  the angle that the forearm is supposed to be to   connect to the shoulder. So we have this kind of  inverse relationship between the hand and all the   bones connecting to that hand. So let's collapse  this. And with the character still selected,   we're going to go up to animation rigging again  and click rig setup and this will create a rig   for us and underneath this rig we're going  to create a left hand inverse kinematic a   right hand inverse kinematic and a head inverse  kinematic constraints. So right click and create   an empty game object and I'm going to name this  one i k and right and then let's add a component   to this and this is going to be a to bone IK  constraint component. And now that tip needs to be   the hand. So I'm going to lock the inspector and  then click on the right hand. And that'll select   it for us in the hierarchy. So now we can just  click and drag it over to the tip and we could   drag it over the forearm, drag it over the upper  arm, create a target, create a hint, go through   this whole process, or we could just right click  on the top here and click on auto setup from tip   transform. And that will automatically go through  and add the connecting bones. And then it also   creates a target and a hint for us underneath this  game object. And all we have to do is line up the   target in the hint with where we want them to be.  So click on the target and then hold down SHIFT   click on the hand and then animation rigging up  here at the top gives us this cool button where   we can just click align, transform, and it'll  reposition that for us. And we can do the same   thing for the hint hold down SHIFT click on the  right forearm animation rigging align, transform   and then the hint we actually want to be a little  behind the elbow because the hint is telling unity   where the elbow should be pointing. So make sure  you're just selecting hint. And then we're gonna   go back behind and drag it back a little bit. This  will help if the player rotates their hand, your   elbow, you know naturally repositions based on the  rotation and based on the distance from your body,   and this will help with orienting the elbow in  the correct way. And now we're gonna do the same   thing real quick for the left hand. So right  click, create a new game object IK and left and   unlock the inspector and then relock it to bone  IK constraint, let's grab the left hand, drag that   over to the tip, right click auto setup from tip  transform, and then we can click on the target,   hold Shift click on the left hand animation  rigging align transform, click on the hint hold   SHIFT click on the left forearm animation rigging  align transform and then make sure just hint is   selected and we can pull this back a little bit.  And let's go ahead and test this before we do I'm   going to uncheck the XR rig and uncheck the XR  interactor. Make sure you save because at this   point, when I run it, I've had some issues with  it crashing. And I'm not 100% Sure why yet. So   just make sure you're saving frequently. And now  if we go back to the inspector, grab that target   and move it around, our hands should move nice.  And you'll notice that as I get further away from   the hand, it doesn't stretch to meet, it just  kind of points in that direction, which is good.   Now let's set up the head. So still inside of  rig right click create a new empty game object,   I'm gonna call this one ik head. And instead of  a two bone constraint inside of here, we're gonna   have a multi parent constraint, the constraint  object is going to be the head. So let's lock   the inspector click on the head that'll select it  for us or there. And then we're going to make a   target just like the previous ones, I K ID target.  And just like the previous ones, we're going to   select the target, hold down Shift, select  the head, and then animation rigging align,   transform. And then let's drag the target over  here into source object. And this is where things   get a little interesting. This is where we add in  the physicalized hands along with the character.   So the way this is going to work is we're going  to have the mesh hand that is visualized, and that   is going to be tracking the target and then the  target is going to be the physical exponent. So   the target is going to have the rigid body and all  the colliders and then the target is going to be   following the controller using physics. So we have  almost like a three tiered following approach.   So we have the mesh hand that you can see. And  then we have the physicalized hand, which is the   target. And then we have the actual controller,  which is the actual position of the player's   hand. And so all of those are going to be working  together to create a realistic visual and physical   effects. So let's go into scripts and take the  hand script and I'm going to drag the hand script   onto each and target. And we're gonna go through  and mess with these values in here and delete   the animator, because in a video in the not too  distant future, I'm going to do a procedural grip.   And that's going to override the last video where  we did the animator stuff. So this procedural grip   will be able to grab any object and it'll stop the  fingers whenever it hits the surface. So working   on that, but we're just going to disable the  animator for now, because it doesn't really apply   to this particular video. So let's open up the  hand script, I'm going to do two things in here,   I'm going to go through and comment out all  of the animator stuff. And then we're going   to fix the hand tracking a little bit because in  the last video, I was adding the offset in local   space instead of worldspace. And that was kind  of creating a weird pivot effect where your hand   wouldn't pivot on the correct point. So we're  going to go through and fix that. First, let's   comment out all of the animator stuff. So coming  out the require animator component, I'm going to   highlight this chunk of animation variables,  press Ctrl, K, and C and then I'll comment   out the whole block and then the same thing for  the start method. And then the update method.   And then each one of the methods below. There  we go. And then here on line 59. This is where   We need to set it to a world position. And in  order to do that, we're going to do a follow   target dot transform point. And that'll transform  from a local space to world space. And then we're   going to put the position offset inside of that  method. And that should fix the rotation pivot   point problem that some of y'all have been  experiencing, make sure to save that we're   gonna go into hand controller now. And you'll see  down here in the update method, we no longer have   a set grip or Set Trigger method. So I'm going to  comment both of those out as well, at this point,   realistically, you could just delete the hand  controller method or not using it at the moment,   make sure to save both of those, I'm gonna go  back into unity, we're going to create a new C   sharp script, this one's going to be called a  head and head is going to be attached to the IK   head target. And let's open this guy up similar  to the hand controllers in this controller,   we're going to update the head to follow the head  mounted display that the player has. But we don't   want physics to be involved in this, we're just  going to have it instantly update. So we're gonna   do that. And then we're also going to update the  whole body rotation based off of the head in this   class as well, we're gonna need a few serialized  field variables to transforms a root object,   this is going to be the character and then we  need a follow object. And this is going to be   the head mounted display. And I can declare both  variables without having to write the line twice   by just doing comma the next variable as long as  they're the same type. And then we're gonna do   the same thing for a serialize field private  vector three, we're going to have a position   offset and a rotation offset, there we go. And  then we're going to need one private variable,   this is going to be a private vector three,  and this is going to be a head body offset.   And the head body offset can be set inside of  the start method. And that's just going to be   the difference from the route object position  and the follow object position. So in this case,   I think the actual position of the character is in  the center. And then the follow object position is   going to be obviously wherever the head mounted  display is. And so this just accounts for   that difference, and will automatically set it for  us. And then we're going to change this update to   a late update. And first, we're going to set  up the root object position. So we're going to   set the body and that's going to be equal to the  Transform Position, bless the head body offset,   and then we're going to do object dot forward is  going to be equal to a vector three dot project   on a plane, and then we're going to use the  follow object.up and a vector three. And that's   going to be normalized. So this accounts for the  offset between the head mounted display and the   body and will essentially set us to the correct  height and then this guy will set the correct   rotation for wherever the head is pointing.  That's also where the body is going to be pointed.   And now we can set the offsets for the camera. So  let's do transform session is going to be equal to   follow object dot transform point, just like we  did over in the hand class. And we're going to put   the position offset in here. And then we do the  rotation offset transform rotation is going to be   equal to follow object dot rotation multiplied  by a quick carnian EULAR rotation offset, make   sure to save and let's go back to Unity and set  up all these values. So inside the head target,   the root object is going to be the character the  follow object is going to be the XR rig which   we can re enable and the main camera was going to  be the follow object. And then if we go up to the   hand targets, we can set the follow up objects  for those so the left hand target follow object is   going to be the left hand controller and the right  hand target follow object, it's going to be the   right hand controller, make sure to save save,  save, save. And now that the animator has been   disabled, we can also remove those from the hand  target. And now all that's left is to run it and   let's set the offsets Oh, and we forgot to add  the you know, physicalized hand part. So go into   your right hand target and we're going to add a  rigid body make sure to set to interpolate and   continuous and don't use gravity. Same thing for  the left hand. And then underneath each of these   we can create a new game object and this is going  to be the collider let's add a box collider here,   we can resize and reposition that and you  can add as complex colliders as you would   like and I'm going to duplicate this and drop  it into the left hand is reset the transform   and then reposition that the head body offset auto  setting has given me a little bit of trouble. So   we're gonna go back into head so instead of having  this start method here, I'm just going to delete   the whole start method and gonna put the head  body offset in with the other serialized vector   threes and rename it without the underscore and  now we can manually set it and we won't have any   issues. There you have it. Now obviously the  legs look kind of weird just floating around   being stiff and straight. So if you want to just  get rid of that As you can find the legs inside of   the bone structure and scale those down to zero  so that they disappear and you're just having   a torso and arms and hands. In an upcoming  video we're going to work on procedurally   walking and having those legs animate. But  for now you're just a floating body,   I want to give a super special shout  out to our newest VIP Patreon. Scott,   thanks so much for joining us. And I really  appreciate your support. And to you, Eva,   Jane, Tim and Austin, thank you so much for all of  your support. And I'm thinking instead of a live   stream this month that we're going to be doing a  private zoom call on this coming Sunday. So be on   the lookout on Patreon for that the time details  and the link for that will all be up there.   And if you're not part of the VIP tier and want to  get in on a private zoom call with me this Sunday,   be sure to sign up on Patreon or if you just  want the source project and all the code that   goes with it you can support me on Patreon and  get that as well and free of charge. As usual.   I'm always in the comments. So if you ever run  into bugs or issues or improvements or thoughts   that you have, be sure to write a comment and  I will respond I respond to every single one.   I love hearing from you guys and honestly all of  you thanks so much for the likes, the comments   the support, the amount that this channel has  grown in the last month has been absolutely   ridiculous. I am so appreciative of all of  you and I will see you in the next video.
Info
Channel: Justin P Barnett
Views: 38,372
Rating: undefined out of 5
Keywords: Justin P Barnett, unity xr, unity new input system, xr interaction toolkit, unity vr tutorial, beginner, vr inverse kinematics, inverse kinematic, inverse kinematics unity, unity ik, unity ik tutorial, inverse kinematics tutorial, openxr tutorial, openxr, vr inverse kinematics tutorial, vr ik tutorial, vr upper body ik, vr upper body inverse kinematics, unity vr tutorial for beginners, unity vr game development, inverse kinematics explained, how to do inverse kinematics, 2020.3
Id: MYOjQICbd8I
Channel Id: undefined
Length: 16min 38sec (998 seconds)
Published: Wed Mar 24 2021
Related Videos
Note
Please note that this website is currently a work in progress! Lots of interesting data and statistics to come.