How to 3D Motion Capture Easy with just a Phone!

so in the past if you've wanted to get into motion captures either gonna cost you thousands of thousands of dollars or you could use some sort of janky system with like an Xbox Kinect as some third-party applications and that's kind of what I thought was the only options available until the company radical reached out to me and they wanted to do a sponsored video on their new app that allows you to motion capture data right from your phone and then upload it to an AI system that tracks your footage and extracts the motion capture data right out of it for you and I'm like wow so then I downloaded the app and I tried it and now it's like that's right guys in this video we're gonna be using the radical motion app and doing some tests to see what kind of results we can get we're stopped by shooting some footage having a motion captured and then we're gonna take that motion capture data and port it into blender and use it on a 3d model to get some animated results so if this works well you can definitely use this for many applications like game development animation blocking and like reference and stuff there's so many applications that I can see this being useful the app is currently running their gen 2 AI software which is pretty good but there is actually a gen 3 in the works that is coming very soon there will be a desktop app for your computer and would be even more powerful with even more advanced 3d motion captures so that's really exciting so go ahead and get the app you can use the link in the description below and that will save you 10% off the subscription but there is a free trial as I say in the title of the video so you can go ahead and download the free trial and get some free upload time and download time to play around with the app and see if you like it once you use your three minutes radical offers many different affordable pricing options to accommodate to any budget and anyone that signs up for a year subscription will automatically get their membership renewed for free once Gen 3 is released in the next month or two so once you have the app installed you're gonna create your free account go ahead and sign up with an email and password and then you'll be able to log in right off the bat they give you a few tutorials to get started with the app pretty basic things but I'll help you get better results and I'll also be going over a few of these tips in the video so now for filming the video we're just gonna use the in app camera and obviously you gonna want to do it in portrait mode although it's not required to shoot footage within the app and any clip captured from a consumer-grade camera will also work you'll also want to lock your camera off on a tripod for this I kind of rake something up on my tripod just to hold my phone as I didn't really have something sitting around as long as it's not moving you're gonna get better results and then for your motion capture you're gonna want to wear some tighter clothing just so you don't have too much baggy clothing as well as wear something that contrasts from the background a bit so you don't want to have colors blending in with the background but as long as you do those things and keep your body in full frame of the video you'll be ready to go with some motion capture so no just going ahead and starting a recording you can set the timers do you have some time to get in front of your phone and you don't want to start off hold eighty pose as this will allow the app to kind of track your limbs and get the rig set up correctly and then go ahead and make some goofy wonky movements making yourself look like a fool in front of the camera giving it some information to track and then once you've completely made a fool of yourself by dancing around in front of your camera you can go ahead and upload that data to be analyzed this will usually take twenty to thirty minutes and if you go to the website online you'll have the progress bar there of how long it will be until the footage is motion tracked and are ready to be used so after about 20 minutes our footage is now analyzed and you can see what it looks like here in the web browser you have a nice app to kind of see how your scan worked and as you can see this one actually worked pretty well we have some nice camera tracked movement and this should definitely be usable so now that we have this you can go ahead and download the FBX file right from radical they're a very small file and we're gonna be ready to happen to blender and open up this motion tracking data on one of our characters I'm gonna put the link to the original creators of this a robot character that we'll be using in the description but you can also download the version that I have in the description that is poor Eevee which will be what we're using in this video the one other thing you'll want to download is there is a tipos rig in the description that you want to also download we're gonna use this rig to map it to our character so that rig can then follow our motion capture rig perfectly so go ahead and download both the rig and the character from the description and we'll be ready to roll so here we are now in blender 2.8 and we have our cute robot model being rendered here in a V I just laughed some quick basic metal materials over this to work in Eevee didn't go through too much time but it'll be perfectly fine for the the video here so I'm just gonna go over and switch to solid view and we're gonna open up that rig so we can map it to our character here and start having it animated to our motion capture data so going over to file and append you just gonna want to grab the rig T pose that I give in the description then go to the collection and open up collection one so here you'll see we have the rig and we also have another part of the rig that is for the AI K so just opening up the collection over here you can see we have the reference rig in the I can rig I don't want the ika rig so I'm just gonna hit X and delete that then with our reference right here selected we're just going to kind of line it up over our character I'm just going to grab it along the y-axis here and kind of line them up closely to the character and then if you go over to your viewport display here over on the object down to the viewport display you can enable in front and I'll make it so you can always see your rig nicely here so with our rig placed right about there we'll just kind of get the arms set up and that's the most important there we're going to kind of start placing the bones over the limbs of our robot a little bit better so we can have a little bit of reference of what's gonna be attached so first off when you go into edit mode on our rig here and when I choose X access mirror over in our properties tab here under tool just choose x-axis mirror and this will make it a lot easier so we only have to edit half of the mesh and I'm also going to start off by deleting these fingers off of this rig as we don't need the finger capture data as the app isn't really good enough to capture any finger movements as that would be literally insane so go ahead and delete that we're just gonna leave the arms there and then for the toes we can also delete the last bone there as we only need one bone for the feet okay now I'm going to go ahead and start placing our bones here actually this one's not connected so if you have a bonus not connected you can go over to your bone settings here and then just click connected and now you can see it moves with it so I wouldn't do that the same thing for the foot there is that's not connected either and then we're just gonna line up our bones sort of to our robot mesh here so I want it to be hinged around that that ball shape there so we're just gonna do that and then grab the toes pull those over as well kind of lining up the bones with our character and then the bones here can be centered right over there I figure the side view we have to do a few more options on our our bones here just centering them up over the ball joints on the robot here pretty simple stuff nice and easy next up was line up the bones for the arms so the arm socket here we're gonna want to be right about that point there and if it helps you can go to Z and go to wireframe so you can kind of see the joint behind it there I want to grab both of these bone joints by shift-clicking in both of them and then you can just place it right over the hinged area there same thing with this one grab both of joints there place it right over that ball joint there and then the the end of this would just go to the ball joints on the hand perfect all right now when we go to top view and line these up from that view as well grabbing on the lung x-axis here or the y-axis I should say and lining them up right along our rig the last thing to do for setting up the bones is there's kind of three different parts to the robot here the bottom the center in the top and you just kind of want to line up these bones with those bones so I'm just gonna kind of pull this one down a little bit here and then pull this one up a little bit here just so it kind of fits that stomach there and that's really all you have to do so now with that said and done we can start mapping our rig and hit Z and go to solid view to our character so because this character is broken up into multiple pieces you won't be able to use the automatic weight feature to assign the weights but that also kind of works because this is a robot and so the automatic one wouldn't really work for assigning the parts anyways as you'll see in a minute here so just go ahead and grab the mesh and then shift grab your armature and go control P with automatic weights even though it won't work it will help to assign the groups so this will do some of the process for us and after about 10-15 seconds it finishes but says that it couldn't complete it because of the issue I just pointed out there's too many loose parts on this robot if I tab into edit mode you can see that there's all is individual pieces and that is why the automatic weights won't work but that is fine because we're gonna take our rig here map it to our character and then open up our motion capture data and assign this rig to follow the motion captured rig this might be a good time to mention that the next update to the app featuring the gen 3 AI is also coming with an improved rig which will make it much easier to assign rigs to characters in the future and there's actually going to be an add-on developed for that too so some super exciting stuff coming from radical so after parenting it now with automatic weights if we grab a rig and go over to the vertex data here you can see that we have all the bones in here and this is what we use to assign the different bones here now to the different parts in the mesh something else that will make this process even easier and if you grab your rig and go over to the under the armature settings here and the object of data you can choose names and this will allow you to see the name of every bone and we'll make it really easy for assigning the bones to the mesh here now so I'm going to tap into edit mode and I'm going to select everything that I want this part of the arm to be assigned to I'm gonna choose this option here which allows me to select vertices high in the mesh that I can't see so I'm just gonna hit B and box select all of those vertices and then I'm gonna hit L to select the rest of that part there and you can see that gives me that arm joint which is exactly what I want and I want to be assigned to the left forearm so originally grab left forearm over here click assign and now when I move that bone if I tap out of edit mode grab that rig here go into post mode you can see that that arm is gonna move now with this bone which is exactly what we want so this is just the process that we're going to go through now and add the rest of the weights to the different parts of our rig here so for the left arm here and I can just hit L select all the different loose parts and once I have them all selected then I can just grab left arm assign and that's gonna work great jumping over to the right arm here we can make sure that this is the only thing we have selected select all those loose parts grab the right arm here and click assign so this takes a few minutes so I'm not going to show the whole process but it's basically just grabbing the parts in the mesh that you want to be assigned to the bone and then going into to the settings here and assigning them pretty simple stuff [Music] now for the body here I'm just going to assign the bottom here to the hips and the spine as there's more bone to that I really need so I can assign both the hips and the spine to this bone then I'm gonna grab the middle chunk here by hitting Alex selecting it all I'll also gonna have to select this little part and back to the head here our back of the chest there should say the back of the body I guess would be the right way to say it and all we're gonna do is assign the spine one to this we can also assign the spine tube as again they're both bones in this group so just assign spine to we're doesn't hit L and select the chest here and just assign the right shoulder and the left shoulder both to this bone now for the neck we're just gonna hover over it hit L and assign the neck to it everything is assigned so I'm just grabbing our rig here jumping into post mode and I wouldn't do a little bit of moving around here and making sure that everything works the way we would expect it to so far so good everything's moving around the way we want it to and you can see that we did miss two parts here that should be assigned to the chest so I'm gonna talk to object mode and that's just these two rubber pieces on both arms here and we're just going to assign those to the spine too and also to the right shoulder and left shoulder but without a sign now it looks like everything is working correctly and we can go ahead and map this character to our animation so this is the fun part guys we're gonna go ahead and import that data that we just captured and when I map our robot to it so to do this I'm gonna go over to file and I'm going to go import FBX here I'm just gonna locate the Android scan that I just downloaded that I captured off my phone and import the FBX file this will take a few seconds to import and there you can see we have our data so it starts off with me waving like I did moving around kicking my leg back moving my arms around and all the data looks like it's there and looks like it's actually captured really well so let's go ahead and assign our robot to it so the way we're going to assign the robot to it is we're going to jump into post mode and as you can see all these bones already have a constraint added to them and that's because we're going to be using a constraint to map this bone to that bone and vice versa since we're using the same rig on the character now as we're using on our motion capture rig so I'm just going to grab our left leg to start jump over to the constraints here and all I have to do is grab the armature for this rig which is if I tab into object mode here grab it you can see reference does 0 0 1 so all I have to do now is grab our bones here jump to pose mode and with this bone selected I'm going to choose that armature and it right away assigns it to the right bone as well so it has the right rotation and angle on it and it's actually working just the way we'd expect it to so we're gonna go ahead and do that for the left leg up and all of these all you have to do is punch the rig in there you don't even have to do the individual bones and it will snap it right to it you want to make sure you do it for both the legs here and you can see that those are already in their right position very cool and we'll just go ahead and do it with the arms as well all you have to do is choose the reference rig 1 and it will automatically assign it to it [Music] and if I was to play this back through here you can see that our character is now making the exact same motions that we just motion captured through the app really cool stuff but you might notice that because of the robot shape and because we had a cop distort the rig a bit for this character some of the bones aren't really lining up with the joints that they should be at if I click that button you can see clear that they should be over here and it's offset a bit so this can actually be fixed tabbing into edit mode grabbing your bone joints and just kind of moving them at this point if you want to try and fix this so for example if I move these in a bit closer you can see that that kind of gets the bone a bit closer to where we want it and so if I move it in even closer you can see that the bone is closer to that joint so this is just kind of offsetting it now by moving them around and this is also really only necessary because of the character that we decided to use and it being a bit different than the rig kind of expects it to be because it's not really a few minutes robot and so joints are a little bit different than they should be but there you go I just move it around a bit and got it lined up quite a bit better in relation to where it should be there so that's looking better for the legs that's another thing I've noticed with the feet that they don't really work to track how much the rotation kind of gets messed up I'll show you here in a second if I was to grab the foot here and track it to it the rotation kind of gets twisted so for whatever reason the feet don't really work and you just want to leave the foot assigned to the leg kind of similar to the hands so do the robot shape we don't actually need all of these bones assigned here and actually taking the ones that are duplicates off will give you a little bit better results so I'm just taking the constraints off of some of these and you might also notice that the head is kind of low on the neck here and that's just another reason why you might want to take the bones here down a little bit and the more you pull them down the more neck he'll kind of have there so that's kind of helpful as well just to kind of bring his head back into the right place so I just shifted around a few of the bones here in edit mode just to kind of make him match up a little bit better with our rig here as you can see and it looks a bit better throughout the animation here now it's not going to be perfect due to the rig that we chose if you have that was all attached there'd be a little bit of stretching but it would might look a little bit better but you can see we have some really cool-looking results here and if I switch to Eevee it would get even cooler so here you can see our character and I can just hide the rig here if we want and here you can see our character moving around just like I did in our video now there's gonna be a few errors with this but it's pretty fantastic there's a few errors mainly because of the robot that I used you can see that because of the joints not stretching do the separate parts you'll have a little bit of areas we have some mistakes there but overall pretty cool stuff and if you want to use like the low poly character from my tutorial from a few months ago you can get a result like this I'll show you right here and this was just done using the low poly character and you can see when you have a mesh like a human and it stretches that hides some of those issues even more so but pretty cool stuff this is a lot of fun to play around with it's also super cool to go ahead and then render it in Eevee so you can have it played back in real time animation here so how cool is that to be able to have the animation then being able to be played back here with Eevee being rendered in real time so some pretty cool stuff and I'd like to again thank radical for sponsoring this video with their app really cool I highly encourage you guys to check it out if you're at all interested in motion capture this is a lot of fun to play around with and again if you use the link in the description you'll save 10% when you sign up for a subscription that gives you a lot more download time a lot more upload time so you can do a lot more with the app but that's gonna kind of wrap up our video guys that is motion capturing with nothing but your phone really crazy I'm a really impressed with radical um I hope you guys have some fun playing around with motion capture if you want to use the motion capture data if I haven't mentioned it yet here you can use the link in the description where you'll find this file you can download online to motion capture the same footage are used but yeah it's a ton of fun I highly recommend you guys try it out and I can see this being really useful for a lot of things like I said earlier game development for one I've already heard of a few game companies that are using this for development as well as like mapping out animations and you can even use this as like you're blocking animation and then just come back and maybe tweak it improve a little bit where there was a few errors but like this this does most of the heavy lifting for you so pretty crazy cool things being done with the radical app super exciting and I hope you guys have some fun too playing around with it also a new video that's going to be coming soon is gonna be taking this character and its motion capture data and compositing it into live footage so adding 3d characters to live footage that's gonna be a new tutorial coming soon but in the meantime guys have some fun motion capturing yourself and putting your animations onto characters it's a ton of fun trust me so that's one do it for me though I like to again thank radical for sponsoring this video super awesome app super cool things coming and I'll see you guys all in a future video bye bye I'll see you later ba ba

Channel: CG Geek

Views: 946,204

Keywords: Motion, Capture, Blender, Free, Easy, Learn, 3D, CGI, VFX, Tracking, Phone, Beginner, How to

Id: fwc9vTCLYCc

Length: 19min 59sec (1199 seconds)

Published: Mon Jul 22 2019