Realtime Motion Capture iPhone to MAYA

[Music] hey guys well we're on the mocap EXO website and as you saw in the demo just now and on their website and you have a full control of your rig character by using an app on your iPhone how cool is that right now I did a bunch of features I'm going to go through I'm going to show you how you can set this up I'm going to show you how to use the demo character but more importantly I'm also going to show you how to bring in external models that are rigged even your own and how you can do exactly the same thing right so let's set dive in and check it out Hugo okay everybody well as you can see or in Maya and we're in Maya 2019 to be exact reason for that is that they're working on the 2020 version and I don't want to risk it not being stable yet or anything I think it's an beta but 2019 works perfectly fine now what we need to do first is we need to make sure that the mocap x plug-in is activated but before we do that I want to give you a quick summary of what's going on here okay so as you saw in the intro clip there is an app that you can install on your iPhone it's called mocap X it's a free app and it will allow you to communicate real-time with a character in Maya basically you move your face it will move on the screen of your cell phone it will then communicate to the character in Maya and that character will move in the same way at the same time you can then record that information and then it will be all keyframed and then you basically have a out say a primary setup for your character animation you can then go in and tweak it and you know work in the graph editor and so forth so it's a huge huge huge time-saver okay that said let's make sure we have the app loaded I already see it is up here but I'll just go through the motion to show you guys how to do this so there are clear installation instructions on the website I'll put a link below how to install the app on the phone how to install the plugin in Maya and there also a bunch of tutorials if you need them right okay so we're gonna click on the windows we're gonna go to settings preferences we're gonna go to the plug-in manager and I'm going to search for a mocap right and there you go add the mocap plug-in loaded and it's on auto load so it will load every time okay so we got that going on now we're going to do next is we're gonna open up our demo head and you're probably wondering okay do I always need to work with this demo head no of course not you can create your own models your own rigs and then you can set up the links so you can control all of that but for now we're gonna open up the demo add I'm gonna double-click on Natalie right here actually just click once that's fine and there you go now the cool thing about Natalie here is she has already been fully synced up so I believe it's this guy right here yeah the post live editor so if I open that up you'll see there's a whole range of stuff that has been set up for her specifically so if you have your own model and your own rig what you can use is this as a base reference right so you can go in here and you can select one of the motions here mimic that on the the rig that you have and then that will synchronize with the phone kind of sounds complicated but I'll show you in a bit right okay but for now let's make a connection so we've got Natalie going on here you can control the rig straight up if you want but that's not the point and we're gonna do next is we're gonna set up a connection now this is a little iPhone symbol here before you click on that make sure that the app on the phone is running and make sure you're in streaming mode because you have to be in streaming mode to set up that direct connection okay so I set the phone up it's a streaming mode what we do next is we're gonna click on this little iPhone right here and I think it's open down here but let's try it out again yeah and then we're gonna open up the attribute editor and see if that was successful okay in the attribute editor we now have a tab here where it says real-time playback controls and we're all gonna do is we're gonna set up a connection with the phone now you can do this with a Wi-Fi connection or a click here and use a USB connection and the USB connection is typically more stable so that's what I would go for and I'm just blurring out my IP address here because you know I don't want you guys to hack my system or whatever so here's the Wi-Fi option here's the USB option yeah and then you're gonna do is you're gonna click on connect we'll give it a sick connection established down here and what will happen right now is as soon as I pick up my phone and start making funny faces and the face ID option in the Apple phone will record that as senator Maya and you should see all the actions that I'm doing being mimicked right here in these bars and then we want to translate that information to Natalie and now we're going to do is you can go up to the top here and when they click on this guy right now this is the mocap post lip editor I'll click on that and then we've got these two little circles right here and that is basically to link them up right so we're gonna click on that and immediately you see the blue bars and little right there so if I now pick up my phone and let me just get some cable out here take my phone and if I look at my phone and starts to talk and move and so forth you see the bars moving based on the controls that have been set up so if I open up my mouth you'll see that Natalie in the example is doing exactly the same so we've got a movement we've got eyebrow movement we've got jaw movement mouth movement so forth pretty cool right so that's how that synchronization works now let me just put the phone away as you can see as soon as we do that everything kind of calms down a bit and then we're gonna do is we're gonna see if we can do the same thing but with a different model than the demo head now for that purpose we're gonna download a free rigged character and I'll show you guys how that works okay guys well we're on a website called animation studios calm and that's the website or we're gonna get our rig for our demonstration so we're gonna go to the download section and then we're gonna go to the example rigs and we can choose one of these right here and it doesn't really matter which one you choose but we're gonna go with the top one I think this model is called Claire there you go it's a zip file so what I'll do is I'll unzip that using unzip software and then we'll open that up in Maya right here we go okay guys as you can see ramaya we're still in my at 2019 and we're gonna do is we're gonna open up our when we just downloaded so I'm going to go to file we're gonna go to open scene and here is Claire okay open that up there we go loading that up and boom there it is alright we're gonna jump in to our perspective mode I mean at 6:00 to apply the textures and the idea here is that we are going to hook this rig up to our mobile app so we can control it that's the idea right so we need to do first is we need to drag select the controls the face controls like that okay and we need to create an attribute overview so I think it's called attribute collection there we go when I click on that right that is now empty just yet but it's okay that's we're gonna work with and we're also gonna open up the sky all the way to the left now that's the pose lip editor yeah and we also need this this is basically the Natalie reference board like what I like to call it it's the mocap pose board and it'll become clear in a second right okay so if we look at the right hand side here and we open these up for example we have the left eyebrow going up and down right in this case down now that is a movement that is linked via the rig to the application on the phone now we want to do the same movement on this character and kind of link that together right so what I would do is I would select the left eyebrow right there right and what I would do is I would move that down like that for example yeah and then I would simply click on the + next to the eyebrow down and make sure we have the left one yeah it's so simply gonna click right here as we do that you see that this has been added right there which is exactly what you want now let's say you want to do something else we want to drop the jaw so we're gonna click on the jaw control I mean it II I'm gonna rotate that down kind of open that up like so until I'm happy with the pose then over here in the pose board we're gonna look for jaw down and let's see if I can find it jaw open that's what it's called right just click on this plus sign so you can see what's going on here that's the one we want we're going to take on the plus sign it's going to be added here as well now this is what you do going all the way through the model so let's do one more let's see well do I don't know we'll take them out the corner right there so we're just gonna add W and we're gonna kind of move it up like that right ok so let's see if you can find that here as well probably under mouth and I assuming let's just look here yeah that will be it we're gonna go with that one ok so we have these three now you're gonna run through your entire rig that way right and you only have to do that once once you've done that you have that control and then you can control it remotely so we're gonna get rid of this we're gonna minimize that okay and now with this setup we're gonna establish a connection with our iPhone so we're gonna click on the phone symbol up here we're then gonna hit ctrl a to open up the attribute editor and here you go here is the connection again it's set to Wi-Fi I'm gonna set it to USB I'm gonna make sure that my application is running and that it's actually in streaming mode which is kind of important I'm going to establish the connection connection established as you can see down here it's not responding just yet in order for us to get that we need to link this up so hit these two little circles there you go so right now you see that the mouth is moving the eyebrow is moving yeah and look at the corner of the mouth it's moving a little bit not a lot I'm making some weird faces here but you can clearly see the eyebrow moving in the jaw moving that's kind of the idea right okay now let's say we have the entire mom rigged up and it all looks great and we're all happy and all done yeah the next thing you want to do is you want to get this thing key frame right okay so what I'm going to do first is I'm going to record an animation that I like now they did a kind of smart thing at mocap X because normally when you start to record the animation that you get is not the one you wanted you have to go back and do it again and again and again now what they did is they set up a way where you can record backwards so right now in this tab here it says record controls and I'm first going to check a location where I actually want it to be saved because I don't want to have that funky path so it's basically just going to be there I mean I think it save as Claire so when I start to record this it will be saved as a file called Claire on my desktop right now instead of 30 seconds I think something like 10 or so would be okay and I'm gonna keep on talking so my character is responding and I'm simply gonna click on save clip alright so it went back 10 seconds and recorded what I did the last 10 seconds okay now I'm just gonna put the phone away for a sec and if I now move over to the next tab and go to mocap X right here I have the option to open the clip reader when I do that just above here and I have the real time device which is connected to my phone and I'll just show you one more time there you go and you have this one which is the clip reader now that's the one I want because I want to load that file so click on the clip reader click on the folder we're gonna go and find Claire and she will be on my desktop there she is and we're gonna open that up okay now you can see it's processing and let me just tweak the number of frames here let's set that to 300 so we can see what's going on yeah and you can see the mouth moving alrighty cool cool okay let that run all the way through okay and now you want is you want them to stop this what you want is you want to bake this say have P frames okay so in order for us to do that what I'm gonna do is I'm gonna drag select those controls again and just try to imagine that you have all the controls done right I just did a few but you have all of them none so I've got this selected what I'm gonna do next is I'm going to go up here and I'm going to bake them so I'm gonna click on bake it's gonna start to run through the keyframes and you can see what's going on right there you see the mouth moving all right and now you see down here is you have all these keyframes that have been created right now to be honest I think when I baked it I only selected the mouse control and not the others that's why you only see the mouth moving but it's the principal accounts right so make sure you got everything selected now all of these keyframes are selected now let's imagine that you have to do a nomination for a client and you use this mocap but you don't want to client the see that you use the app right once you're at this stage you basically have a keyframe animation like any other so you can go into your attribute editor and you can get rid of everything that has been created for this so the pose lived the mocap as long as you keep your group you're good right and then you know you can just say hey this is a keyframe animation that's all this to it so hopefully that gives you a better understanding of how mocap x works keep in mind I am NOT an animator so if you are you understand this stuff way better than I do I think it's a pretty exciting combination of apps and plugin and I'm certainly gonna be playing with this more in the future and I suggest if you like this kind of stuff you check out the links below I'll put a link to the website to the app and basically everything you need to know okay well thank you guys so much for checking this out don't forget to subscribe if you like the video and see you guys next time bye [Music] you [Music]

Channel: MH Tutorials, the Modeling Hub

Views: 43,349

Keywords: mike hermes, mh tutorials, mocap, motion capture facial, mocapx, apple app store, face tracking, facial tracking, face id, maya 2019 tutorial, maya mocap, maya facial mocap, motion capture animation, motion capture app, motion capture tutorial, controlling rigged models in maya, learning maya 2019, how to become a 3d artist, maya mocapx, maya face mocap, maya mocap files, maya mocap library, mike hermes donut

Id: j_0W5_QlOH8

Length: 16min 50sec (1010 seconds)

Published: Wed May 13 2020