Ep.2: Real-Time Facial Tracking using a Webcam in Unreal Engine 5

wow this is really [Music] working hey everyone welcome to this new episode of my journey while I will try to create the best possible virtual assistant based on a metahuman in a real Engine 5 in the previous episode I was tracking and animating the entire body of a manne using a simple webcam if you haven't watched it I will put a link in the description below in this video I'm tracking the facial features of a metahuman I've created a new project went to the quicksell bridge chose meta humans I picked a meta human downloaded it and added it to the project I will control its facial features using a regular webcam in order to achieve that I used the lelink protocol to stream facial capture data into Unreal Engine but first I needed to install the correct plugins I went to edit plugins search for a kit and selected the plugins I also open the Livelink panel by going to Windows Virtual production live link but you have probably noticed that these plugins work only with an iPhone app called Livelink face however since I cannot afford an iPhone I have decided to come up with an alternative solution I have implemented an app that can turn an ordinary webcam into an iPhone so here is the app and as you can see it has already detected my webcam and will do its magic that's why I called it magic webcam once I run this app I want you to pay attention to this live in panel it will display the device that has been detected and that will be used to control the metahuman face so here we [Music] go I'm sorry Dave I'm afraid I can't do that this conversation can serve no purpose anymore I hope you like this video if you did please press like And subscribe to be notified of the next episode thanks for watching and I'll see you in the next video

Channel: Samir Manar

Views: 3,772

Keywords: Unreal Engine, Webcam, Mocap, Motion capture, C++, Mediapipe, Yolov8, Pose, Tracking, Real-time, facial, IPhone

Id: u3kjn-Zw-Mg

Length: 2min 51sec (171 seconds)

Published: Sun Nov 12 2023