Facial Motion Capture with Blender

paper on blaze Raider here again today I had this idea to test out my way to do facial motion capture with blender and the new tracking or was new tracking system and here is the result after 1/2 hours so I've got these individual markers on my face here and I've used blender to track those markers and apply the motion to the face model you don't mind the funky face paint over here is just for testing purposes so you can see there's some issues with the mouth here but this is just after 1/2 hours of work and not refining the whiting of the the mesh at all mainly just um try and get the system to work so I'll show you how that's working now so on this this head model here I have a mesh deform modifier and just the subsurf modifier to smooth that out and if I go to fight on idea you can see this um this cage that I've got here and that on this layer I'll show you this so this has arm hooks applied to each vertices here which is tracked so we've got 28 of them there's 28 track markers here and each of those track markers are vertices move around by the actual tracking data so you can see those are the if I go to come with the camera view here so these empties follow the track and so they've they've been constraint to do so by by using the tracking system here and each one of these vertices controls not virtually sorry each one of these empties yeah so this just takes one for example has its movement applied to these vertices so just to bother you might notice that they're a bit out of place when I go into edit mode that's because I'm the hook sound applied actually they're not active while I'm in edit mode so this is a hook so this some controls movement of this vertice as well while it's in am as a mesh object so as that those are hooks deform this object here with mesh of this over you can see it's following my facial movements here so as that moves around eat all that movement is also applied to my face here now this is just really rough you can see there's a lot of issues going on here it's not even meat accurate the eyebrows and stuff they look ok each at the mouth area and the nose requires a lot of moving now take into account also that my head is not completely stabilized I did try to stabilize the footage as you can see here is that Alright I had the original footage he must have gone after I saved file or something but you can see that the footage is not stable my head is not completely stable I've tried to stabilize it I told you already the background is moving but my head is still slightly moving up and down you can see from my facial region here and that also applies now we have some weed movements here and if I go to just the side view here you can see my chin is not moving the way it should be now this is a problem with the the mesh deform modifier if this um this head model here or let's just say yeah the head model here this how to correct armature with a bone system and everything like that so one bone for each track here and then it was weighted correctly so you would wait let's say this whole area for this marker sorry this whole chin area for this marker so that as that marker does move it will move the whole draw up and down so that way the system would work much better but I couldn't be bothered going through all that armature making and skinning and everything like that I just wanted to test something quick I just had on my mind thought of it earlier in the day while I was out shopping and I yeah so here is the system applied this might I hope this stud actually does inspire some people to take this further so may perhaps a head mounted camera like like the ones using avatar and one of the more recent movies so that all the head on head moving is isolated just to the face and also our proper bones set up with our mean with our armature set up with bones and some appropriate skinning so there you go it's just a quick visual motion capture test I hope you're inspired to try something like this as well it is a bit tedious to set it all up and you have to really know what you're doing and yeah pretty much know yeah so thanks for watching this have a good day

Channel: blazraidr

Views: 241,003

Keywords: blender, 2.66, tracking, motion, capture, facial, compositing, 2d, 3d, stabilisation, nodes, tutorials, tutorial, beginner, how, to, rigging, mesh, deform

Id: a_AwDv9Al28

Length: 5min 58sec (358 seconds)

Published: Tue Feb 19 2013