Unreal Engine 5.1 Live Link Face Importer

how's it going everybody welcome back to the channel in this video we are going to take a look at how to use LifeLink without actually having the connection number Wi-Fi so this way you actually use this in in a different way instead of being online all the time and uh I should have probably practiced this before but anyways let's get started okay so before we actually get started with the iPhone part I'll show you why I have here what I have here I have my medic human which is going to be the first test then we're going to do a test with a non-metah human character now behind the metahuman you can see a video of myself that I loaded with this same system we're going to talk about why I did that uh in a little bit but just to show you that everything we have here is just you know some uh let me turn things on some lights I have a camera and I have a camera that's uh let me show you its focal length is 80. and I just have very flat lighting because I don't I'm not going for cinematic here this is just for the tutorial sake now before we move into the iPhone part I just want to give some love to my man JS films I actually watched his tutorial and I'm just going to add some things into what he did so thanks Jay for putting that out all right so what you're seeing here is the iPhone trying to capture my face and me behind the camera which is real fun um I just want to show you how does it look so what you're going to see in your face app live link is you're going to go into settings and you're going to make sure that you have the record button enabled because if you don't then how you're going to record that's all the things you need to do in order to do this to make sure that's on then I'm just going to go back and make sure you calibrate before doing anything here and just press record just hit the record button it's the giant red button there and once you are done your recording will automatically be saved to your iPhone so if I get out of this app and I go here into my let me browse on my phone uh LifeLink face in this case because the only thing that I have on this phone it takes and I have the latest take which is the one from the video that you saw in the beginning it's actually this one sorry if it looks a little bit blurry all right so as you can see here I have all the things from my take so I have this one right here is a video and this one right here is a raw file it's a cesv file CSV file you're gonna see that's like an Excel file that you can use with this system but these two are the ones that are of interest to us because we're going to be using the video and the CSV file now how do you get these things out of here well if you're a Mac User and you like iTunes you can just connect your iPhone to your Mac and just download it as you usually would but because iTunes is the program of the devil then what I do is I click and hold and I get this share and then I'm just going to share into my drive you can also my drives right here you can also send it to your email um I don't know what kind of email you use but mine doesn't take 455 megabytes so that's why I had to use my drive for this you can alternately just connect to your computer and just do it that way but the slave which is the important one is actually very small it's 1.4 megabytes so you can send that over email all right and I know my meta human got weird because um while I was recording the other part the live link was going on as well anyways um what we're going to do is we have both the video that I downloaded as you can see for 33 megabytes plus the Slate which my computer recognizes an Excel file for some reason I've looked it up online and CSV is a format that is used a lot with Excel because I think it's a database format but anyways what I mean is this is the file that you're going to be able to drag into your computer to drive your meta human without having to have live link connected at the same time so that's what's the most important thing is you can ask somebody so and for me this is what's really important I can ask an actor to record himself and hopefully with an iPhone almost everybody that I see nowadays has an iPhone like every other person has an iPhone so if the actor that I choose has an iPhone or maybe I can share my iPhone with him or her then they can just record it and send it over to me and I can just add it and correct it inside of unreal and all that so this takes away the necessity of you having to be present and and having to record this in real time so this this was a huge leap when I discovered this it is is great for me to continue doing my short film this way so before this actually works inside unreal you need to enable the plugin so if you go to settings the usual plugins and you just type face import so I just have face info and this pops up so live link face importer is the plugin that you want to turn on uh just check it it's going to ask you to restart that's all you got to do so after that just go to where you have your CSV file located and throw it inside your project it's going to appear as a regular sequence so your everyday sequence there's nothing to it uh the way that I got the video in here was to just right click and as you can see there is a media and just use a file media Source instead of image media Source just use file midi source and uh just double click it and locate your path and that is it once you locate your path make sure that once it's in here a lot of things you need to do is make sure that it plays so the way that you do that is if I go all the way down uh where is it I'm sorry so once you do that make sure that you open media plate and make sure that you load it because it may if it shows in Black that means it's not loaded or something just click on open and it will load instantly so make sure that you open it first before doing anything so I had to reset things to the default so I can show this part so once you load the sequence you can see that there is nothing going on here you need to add a character to it in this case it's going to be automatic human but if you click on this drop down you can see that it does have all the information that you need so all the blend shapes that we usually get with a character or with the Met human are going to be here for you to use and you're going to see that this actually works on characters that have been adapted for LifeLink aside from the meta human so for this to work uh it doesn't work as it is we have to actually add our meta human so we're gonna add our meta human here and you're gonna see that let me uncheck High controls we're going to see all the controls now as usual for this to work uh you cannot have a control rig on or a face control board on so I'm just going to get rid of these there they go and it still doesn't work so what else do we need to do we need to go into the Blue Print go here to details and we have used AR kit phase same with the body so this is kind of like the newish way of meta humans uh doing the anime blueprint because it's already integrated into the meta human blueprint now after deleting the control rigs you have to go here and you're going to see that you have like in my case my name iPhone and over here I do have live link up but you see there's something different about this instead of saying that your iPhone is connected as it usually does it's saying that a sequence live link is connected so actually my iPhone is not on it's the sequence that is reading as part of live link I don't know what kind of voodoo they're having in the background but this is just the way that I think it works uh it's it's kind of tricking the system as if a live link is available so that way when you go into the blueprint of the meta human and you can go here into the arcade phase subject and it's here so if I click on it you can see that I can load it however there's still nothing going on you have to click on use AR kit phase so if I click there you're going to see that my meta human move his head I'm kind of looking down just fine but after that then if I hit play you can see that my meta human starts talking and that is it you can actually record this as a sequence if you wanted to you can bake this to control rig if I right click and I go into actually you have to go into phase bake to control rig you can bake this to control rate to modify which we're going to talk about in a little bit but what I mean is this can be used as a sequence as if you were using Tech recorder so again this is huge because it takes out the fact that we had to use live link then load the source into take recorder then press record then start doing your stuff and I think that that was very cumbersome so this helps a lot by allowing you to do this offline it for me it's it's very big now on to the second part is the reason why my human animation from the beginning looked a little bit better than my usual is because I was able to match it to my video behind me now you see my video behind me is actually not moving and that's because I have not loaded it into sequencer but once I do then I can match my recording which is exactly the same length as The Meta human animation so that allows me to have a reference that I can use to correct The Meta human face I'm going to have another video talking about why you should always do corrections so while your animation is not looking amazing is uh I'm gonna have a whole talk about that so please stay tuned there's going to be the video after this one now we have a reference that we can look at and that we can make corrections to our animation and make it look even better because it doesn't stop at just recording there's always somewhere to do behind that now how do I get this video to run here I'm going to go into the outliner I'm going to get this uh reference video I just load it I'm going to throw it here you're gonna see that nothing happens so nothing's going on and we need this to be loaded in here so if we go into the media if we click here on the plus button we go to MIDI source and we click on reference which is the video that I brought in now you can see that my video is right here and it's kind of starting to move uh all I have to do is lengthening the whole thing all the way to the end and you can see that they are exactly the same and if you spot my face and the metahuman phase uh let me put it like this so you can see it better and if you see me scrolling they move almost exactly the same and this is by what I mean by you need to do some corrections all the time if you want to your animations to look awesome the best thing about this is that I can see my face acting side by side with the metahuman which if you were doing it the way that we used to do it before in order lens in four and five you could not load this this way because you would just do LifeLink and straight up to your computer maybe you just load your video later but it would be a little bit Messier now you can just have this and it's really cool for doing corrections after the fact now I do have an issue um is if I press play sometimes now it's doing it I don't know unreal is it's crazy sometimes it was that if I press play my meta human work star would start going but the video wasn't going at the same time so but if I screw up then the video and the meta human would move so if you encounter that problem it means that the video hasn't loaded yet so give it some time to load hopefully if you have your video in the same folder after project technically it should look faster right now it's loading fast for me even though it's in another address so again I don't know what's going on it's just being weird but if you encounter this problem that it's not playing the video at the same time as the human then what you need to do is just screw up while you're scrubbing it will move at the same time as a meta human now about correcting facial Mo cap I'm gonna talk about that in another video because otherwise this video is going to be too long and I want to show this with a character that is not a meta human to show you that you can do it with any character that is made for a live link all right now what we have here is a alien character that I was using for my other channel I you really need to start doing more content for that channel but anyways now this character follows the live link blend shape guide that you need to have in order for LifeLink to work good with your character so just make sure that whoever does this follows the guidelines they are um there is documentation for that I'm not the one that does the blend shape so I'm not going to be going through that but just make sure that it has the live link blend shapes and you should be good to go now what we're going to do is we're going to load our sequence which I already have right here and the character just added itself I'm just gonna delete this transform all right so I've loaded my sequence and we have our alien character now we have to do is just like with the metahuman we need to add the alien character to this we'll put the alien character a little bit more towards the light there you go all right so we're gonna add the alien character to our sequence and again absolutely nothing is going on you can see that nothing is happening because what we need to do is we need to enable the blueprint that is driving the face motion of the alien to work with this now if we go back to content browser I do have a blueprint already made for alien if you don't know how to create an anim blueprint I'm going to leave a link in description down below where I explain this to the details so you can make one if you don't have one so I just double click the edit blueprint and is this thing right here so because I have Alien made to use my motion capture and live link face at the same time this is the configuration that I have you can also have just one single thing if you are just working with the live link face so in this case as you can see this has this already loaded which is my slave my slate has the iPhone name that I use I'm just going to click here make sure it compiles now the other thing they need to make sure of is that your character is using that animation blueprint so I'm going to load it here and you're gonna see that his face actually changed now if I go into sequencer nothing is happening as usual the reason why nothing is happening is because this is not a meta human I need to add a live link all right let me type it here live link skeletal animation components so it's probably cut in half here but you've seen me done several times just click on ADD search for live and live link will pop up there's live link controller Live Links schedule animation this is the one you want to use so once you click on it now you're going to see that my character moves his face with my animation now this character is not about a human so maybe it doesn't have all the great motion that a meta human has but because it's meant for LifeLink it does work maybe your character doesn't have all the blend shapes that LifeLink requires but if it has some it will recognize this database from the CSV file and it will work so what this means is you can actually use this with whatever character you want all right everybody thank you so much for watching as usual thanks to my patreon they're the level two are always in a shout out on screen you're seeing them right now if you want to help the channel there's a patreon there's a join button uh just follow me on Twitter and Instagram if you want to see more updates about the short film if not then just go to the Discord if you have any questions I'll try to be there a little bit more often I was just I've been swamp with things to do so uh you can find me there if you have any questions if not then there's Kirk and there's a whole community that that it's is willing to help you out please stay tuned for that other video what I'm going to talk about uh something really important regarding motion capture and everything that's been going on and um I'll see you in the next video

Channel: MR3D-Dev

Views: 22,109

Keywords: unreal engine, unreal engine 5, unreal engine 5 tutorial, livelink tutorial, facial mocap in ue 5.1, livelink offline animation, live link face importer custom character

Id: asq5-3FrFUs

Length: 18min 32sec (1112 seconds)

Published: Tue Dec 20 2022