How to Create ARKit Face Tracking for 3D VTubing - FULL Tutorial

foreign face tracking right open blender kill the cube import Mr Baldi in green triangle menu shape Keys copy the name brow inner up and check what it looks like don't worry it'll make sense soon now select your model green triangle menu add a shape key ignore all this stuff I think she used to dance if basis pops up add another shape key a basis is the original model so remember do not select it always keep in mind which Shakey you've selected if you select the wrong key you're in for a bad time rename the new shape key brow enter up time to make the expression edit the mesh and follow Mr Baldy use proportional editing and toggle connected only here and there for optimal facelift results great now we got to do this for all 52 shape Keys oh yeah it's a pain but no one gets into ventriloquism for the adrenaline once you have all 52 the next step is crucial make sure the blend shape names are correct a list of correct names are linked below and unfortunately Mr baldi's defaults are incorrect we gotta rename the ls and R's to left and right thanks Mr Molly we can delete them now remember to check all the blendings one last time if you mock up here Unity will absolutely destroy you export the model and Armature as an fbx good work although we're halfway there the pain has only begun say hi to Unity One wrong step and everything is ruined right click in projects add custom packages we'll need uni vrm BSF SDK and poyon links below after the packages are in import our fbx select the fbx model check Legacy blend shapes apply rig humanoid apply press configure head anime doesn't have I bones or jawbones so remove everything in Left Eye right eye and jaw scroll down done and apply next up materials extract materials give them a new home in the assets folder if they're happy with their housing Arrangement they should auto fill the slots here if you can extract textures do that too or import them manually now drag your model into hierarchy finally reunite the textures with their materials to create some bad CG it's it's temporary I promise okay now we convert it to a vrm select the model in the hierarchy go to vrm export uni vrm press make t pose and fill out the metadata press export and Export it into a new vrm file under assets once that's done hide the old fbx go into the new file and drag the vrm into the hierarchy rename her she'll be an entirely new person 10 seconds later in your new vrm folder go to materials select your material and change it to polyomi eradicated boil me is great only one last thing left the actual face tracking part double-click blend shape to go to the blend shape menu and create 52 blend shape clips it's the last thing I promise drag the preview window out and convert to floating window press create blend shape clip make sure it's the exact same name as the bun shapes we made in blender open the drop down menu and drag the blend shape we want to 100 you know the drill repeat 25 times It's the final stretch done select the vrm model in the hierarchy and go to vsf SDK and Export Avatar bundle to wherever you please congrats unlike a standard vrm a vsf model can have custom shaders cloth physics animations and a bunch of other stuff but that's for future tutorials face tracking we're nearly there Focus buy an iPhone x or above the old ones get hot but the new ones are expensive second hand is good we just want a fancy camera make sure your iPhone is on the same network as your PC on your phone get the app I facial no cap Hey look it's Mr Baldy take note of the IP address at the top you'll need it later on your PC download and open VC face fill in the camera settings with whatever we got the fancy cam at Avatar select our vsf and start go to settings general settings scroll down and check iPhone AR kit tracking receiver type in the IP from your phone if you're lucky the phone will flash green and ta-da your enemy now you may now begin your ventriloquist I mean YouTuber dreams as for the poor thoughts who only saw green but got no tracking whoo go to Windows settings update and security Windows security firewall and network protection advanced settings inbound rules you rule program next browse for vcface.exe next allow the connection next next uh vcface vcface finish huh go back to vcface it should work now hopefully worth it though the tracking is way better than standard visings you can even tweak the blunt shapes to get even better results and now you know how to make AR get tracking oh Mr baldy would appreciate new AR kit users he loves new faces he's harmless mostly run

Channel: Dani Periapsis

Views: 73,525

Keywords:

Id: 19H82IkEx9k

Length: 5min 42sec (342 seconds)

Published: Sat Dec 10 2022