THIS IS HUGE! Everyone can do High Quality Face Mocap now!

[Music] Unreal Engine released a brand new feature for their meta humans which allows us to do High Fidelity facial motion capture and this is pretty huge guys because we don't need expensive gear I mean I'm using my iPhone right now to do this right here it is insane absolutely insane guys so let's figure out in this video how you can do this as well first of all why am I naked It's Not What It Seems in seconds why didn't we have this technology a couple of weeks ago when we were working in our short film this world moves too fast holy land is one of the partners of our short film project that we made in Virtual production more about them later but first let's record some face motion you are gonna need an iPhone 11 or later for this as the LifeLink app which is free from the App Store requires the lighter sensor now I've seen some experiments with only using video but it's too early to talk about that opening up the app selects meta human animator as the capture mode now you can go ahead and record your face for ideal tracking start with showing a neutral face first and optionally one from each side and finally we have the teeth posts stare straight ahead again but this time show off those pearly whites and after that you can do whatever you want with your face make any kind of expression and the cool thing is that it records audio as well which will be processed and unreal so you can use the holy lands Lark C1 which connects to your phone and allows you to record high quality audio straight away now we don't have this C1 anymore since it got stolen so Holly Land send over something brand new and it's pretty incredible guys now they told me not to talk about it yet because it isn't released yet it's a secret but I was able to show it so we show it we show it real good now for audio quality you want your mic to be close to your subject so having such a wireless mic is great more info about this mic very soon guys so if you're looking to buy a new mic just just wait a little bit just hang on it's coming finished recording navigates to the Slate icon on the lower left corner and Export it out you want to transfer that tracking data into the content folder of your Unreal Engine project and you got to make sure that you have Unreal Engine 5.2 installed together with the meta human plugin which you can download for free from the marketplace you will then see the meta human plugin in your plugins panel from which you want to enable it and now we're ready to do some animations now we're gonna need a meta human which you can import from the qixel bridge there are a bunch of presets to choose from or you can make your own like I did I'll leave some links in the description below on how to do that alright download and import your meta human you'll find this back in your content browser to Simply drag the blueprint into your level let's give my meta human some emotions our first task pumping all that juicy tracking data from my iPhone Straight Into the Heart of Unreal Engine now you know me I like to keep my workspace tidy so I'm going to create a new folder called capture Source inside it I'll right click go to the meta-human animator option and click uncaptured source and in the spirit of staying organized I'll name it iPhone double click and here we've got to set the capture Source type given I've already shot my face tracking video and I'm not planning to do a live one I'll just select Life Link face archives next I'll assign the pads where the iPhone data is stored and voila we're done with this window just save it and close now with the capture Source snuck inside Unreal Engine it's time to process it all to get the tracking data or as unreal calls it ingestion and to do that we'll navigate to the tools menu then select capture manager in this new window I'll choose our iPhone capture data and the footage clip or take next I'll select a folder where I want to store the ingested data but I'm going to leave it as it is the next step is to add the take to the queue and then on the right I'll hit the button to import my take once the ingestion process is complete we can shut this window Unreal Engine has now created a subfolder storing all the capture data from my iPhone such as the lens data the frame rate resolution and whatnot now while you don't have to mess with this data it's always good to know where it is tucked away you know during the short film we were able to see a live tracking on a big Monitor and we didn't have to use any cables because the video was wirelessly being transmitted by the hollywoodland cosmo C1 now the C1 can transmit full HD up to 60 frames per second through HDMI or SDI it has very low latency making it perfect for Focus pulling as well and it uses the 5 gigahertz Spectrum to avoid interference and it can transmit up to 300 meters or a thousand feet now holy land has an entire ecosystem for video makers we also used your intercom system the solidcom C1 Pro this was very useful to communicate as we weren't always in the same room plus I couldn't hear a thing in the astronaut suit so the solidcom was very useful there also very useful as part of the costume now what I love about the solid.com is that one of the headsets acts as the master so you don't need a separate station or hop it has great battery life 350 meters or 1100 feet range multiple channels ENC building in incredible intercon system guys and finally we use the Lark M1 wireless microphone this is the same as the C1 instead the receiver can be connected via mini jack to your camera it is super small and it has two transmitters I was able to tuck one away in my helmets to capture clear sounds also comes with a charging case so it's perfect for travel videos as well I used it last year on my trip to the Pharaoh Islands a whole bunch of amazing products guys which I'll leave a link to in the description down below we've been working with the products from hollywoodland now for many years and I can truly recommend it to any filmmaker alright so back to Unreal we've got our Station tracking data first off we're gonna create something called a meta human identity think of it as a dummy or template face match that we can use for our animation now to maintain our neat workspace let's create a new folder called meta-human identity inside this we'll right click navigate to the meta-human identity option and then give it a name and then double click on it to open at the top you'll see a create components button here we'll select from footage followed by our capture data file and now we can start tracking the first mission is to help Unreal Engine pinpoint a neutral face which is the reference point for the entire tracking process with the neutral post selected we'll scrub through the timeline to find the best neutral expression once we've hit the jackpot we'll click on the plus icon button above the timeline this will create a bunch of tracking markers on the face and effectively tells Unreal Engine hey this is your front view and neutral pose oh yeah and also let's not forget to name it although the iPhone does a solid job providing tracking data a helping hand never hurts so let's add frames for looking left and right as well and with these three reference frames and the neutral post selected we can kick start the identity solving process here unreal fits the template mesh to your neutral posts and once this is done we'll rig the identity by selecting mesh to meta human followed by Auto Rick identity skeletal mesh only if a pop-up tells you to select a body type just go to the body tab on the left and make sure pick and then hit Auto rigging again to make the face rig even more precise we're going to add a teeth pose so in the post tab on the left add a new pose scan the timeline for a frame showcasing your Deeds click the plus icon and name it teatpost now one little tweak disable the two lower two to tracking markers on the right sides once done select a neutral pose again and hit the prepare assets for performance button now this will take a while don't worry patience is key go drink a coffee or something and when it finishes you can save and close the identity window now we have a skeletal mesh for our animation that we'll use later let's move on to the actual rigging we'll create a new folder called meta-human performance navigate to meta human animator and selects The Meta Human Performance option now this step will help us calibrate our meta human rig to align with the real life footage creating keyframes for all facial expressions and animations in the performance file on the right adjust the footage capture data to your data under the identity tab choose your freshly made identity next decide on a custom start and end frame we don't need the entire calibration just the acting once you're set hit start processing this will go fairly quick and when it's done it's time to export and the new window choose a location and then a Target skeleton of course this will be the skeletal mesh of the meta human face next hit create alright close the door and now we want to apply this animation to the actual meta human which we can do by selecting the blueprint in the outliner panel on the right and in a details panel look for the face components and set the animation modes to use animation assets under NM to play select the animation and voila we have a stunning meta human animation thank you now if Unreal Engine is still new to you I highly recommend to check out our beginners course which gives you an entire overview on how to create Landscapes working with lights creating Interiors animations and how to set up some of the first virtual production obstacles it's a course aimed at creatives who want to use Unreal Engine for filmmaking link Down Below guys thank you so much for watching thank you holy land for the support and as always stay creative

Channel: Cinecom.net

Views: 311,316

Keywords: motion capture, unreal engine, unreal engine 5, facial capture, metahuman creator, face tracking, metahuman animator, mocap

Id: knf7AH2iqHA

Length: 9min 4sec (544 seconds)

Published: Fri Jun 23 2023