NVIDIA Omniverse Audio2face to Unreal Engine 5.2 Metahuman Tutorial

hey what's going on y'all Jay from JS films in this video we're going to be talking about Nvidia Omniverse audio to face now recently audio to face was updated and I think this is probably the best update we've ever gotten with Nvidia Omniverse audio to face if you have no idea what audio to face is it's basically a way for you to animate characters in this case we're going to be using metamens using just audio I know it's mind-blowing I've made a couple tutorials about it last year whenever it first came out but this right here is very exciting stuff because they made it so easy so that being said let's go ahead and get started I am in the Nvidia Omniverse launcher here and I already downloaded the actual current version which is 2023.1.1 if you don't know where to get that just go to The Exchange and you're gonna download it right here as you can see I already have uh 2023.1.1 I already have that set up but before I had launched that let me go ahead and show you how to actually install the plugin for Unreal Engine 5.1 and I believe 5.2 so to actually really get the plugin started go ahead and go to the library go to audio to face right here click these three lines and then go to settings access the audio to face folder right here and go to UE plugins UE plugins and you're going to see we have 5.1 and we have 5.2.1 so this works with the current version of 5.2 not not three yet because obviously that just came out so I'll double click this and I'm going to copy this folder right here and then go to my project find your project that you're working on and then just create a plugins folder and then copy and paste that folder into here if you want to install this in your actual engine that's fine as well I'll show you how to do that so I'll go to my PC there's all my hard drives C program files and then we're going to go to epic games right here go to the version that you want to install it to in our case it's 5.2 let's go to engine go to plugins and then I usually have a Marketplace here where I just kind of put everything in there and I'll just copy and paste it there so that it's going to be available in our entire engine too which is super cool so I'll minimize this here and I have the project opened right here and if I go to edit plugins since we copied and pasted it straight to the project I should already have that in here so let me type in Omniverse right here and as you can see on reverse audio to face live link is already turned on I have the check mark right here which is good to go so with that being said after the installation is done let's go ahead and launch audio to face so launch this so if this is your first time launching audio to face you are going to have to build the tensor engine so it might take a little bit longer alright so now that we have audio to face open I'm going to scroll down and we're looking for the clear solved AR kit I'm going to double click that and that's going to open it up all right so now that we have both the audio to face on the left and my Unreal Engine 5 project on the right now you can see it a little bit better alright so let me move this a little bit closer because I do need to access this here we're going to go to the audio to face and we're going to untrroll this right here and we're going to go to stream live link now the sweetest part about this plug-in is that we no longer have to really export anything like we've done before like fbx or anything like that we can actually stream this data straight right into Unreal Engine which is freaking sweet using live link live link is amazing so just to make sure that we have live link turned on I'll go to plugins and I'll type in just gotta make sure that we have everything covered everything's good to go it's enabled and as you can see right here is actually here already in my window if you don't know where to get that go to window virtual production and then live link and that's going to open up right here amazing stuff all right so I'm going to scroll down right here so by default this is actually going to play back the audio on your computer and it's going to stream the data to Unreal Engine so what I'm going to do here instead of doing that I'm just going to have it stream straight to Unreal Engine instead of playing back in my headphones because I don't want to get two things all right so what I'll do is I'll click on activate right here and I'm going to say enable audio stream additionally I'm going to my windows and I'm just going to say app sounds and then this audio interface right here I'm just going to mute this because again we wanted to play back only once I don't want it to double up because that would be really annoying and I'll put my earpiece here so I can kind of hear what what's going on alright so the next step we're going to go ahead and go back to Unreal Engine right now and go to Source in video on reverse live link and just turn that on now the audio sample rate I'm not sure what they're using in here so I'm just going to select 44 they might be using 48 on no but I'm just going to set that for that and then press ok so that's going to be turned on and then audio to face I'm going to go to this tab and then in this tab I'm going to change this from Chinese to English let's see here English female we do have a female characters to give me a second English female right here Lindy aec is fine we're gonna have that turned on and as you can see it's going to enable the live link already that means it's ready to go next what we're going to do is I have my medium and selected and in the live link AR kit face subject I'm going to select audio to face that's right here and I'm going to say use AR kit right here and as you can see that character is going to snap now what's cool about this is Again The Meta human blueprint is already set up you just click it change these and you're pretty much good to go so now if I press play good morning Professor Austin how are you doing good morning Professor Austin how are you doing and that's pretty much it that is how easy this workflow is now and what's crazy about this again they also have audio to emotion now and yes it is compatible with a meta-human so what that means is I can actually switch these parameters and you can see it live like you can change how this is being driven from audio to face morning Professor Austin how are you doing if you can't afford an iPhone for facial animation for a meta human additionally if you're trying to create an AI Avatar or a customer service type of Deal or a greeter or a phone or whatever this is the perfect way I think is the best way to do audio to facial animation right now to be honest uh I don't I haven't seen anything as good as this as far as audio to facial animation this is mind-blowing this is insane it can save you a lot of time and a lot of money and and I'm bringing this up because a couple of days ago we had a live streaming in the video Omniverse where we talked about the AI NPC thing which is actually going viral right now everybody wants AI NPCs in their games now and uh convey the company that's making that the company that was doing the demo a couple of days ago um they actually are they are planning on using audio to face with convey to improve the facial animation because you all know I've made a couple of videos about convey on the channel and a lot of people were saying hey the official animation is not that good well you know what I'm looking forward to the day that they can actually deliver convey and audio to face uh with that plug-in in Unreal Engine again that's gonna be freaking mind-blowing and to be honest I think I'm not alone when I say that AI NPCs that's going to be the future of gaming so that's why I'm super stoked about this technology right here and I'm so glad that they're improving it man because I I it's just I mean honestly compared to the video I made I think it was a year ago two years ago when I made a tutorial from audio to face Matty human this is really a couple of clicks away I mean that's really props today in video on diversity and when they're taking feedback and they're implementing that and it's truly amazing but but yeah with that being said what I'm looking forward to is actually Nvidia on reverse audio to face uh taking advantage of the meta-humans facial bones this is using the AR kit but as you all know the medium and face uh actually has a lot more bones inside it if you take a look at it open it up you'll see there's so much going on and I'm hoping that eventually we would be able to get access to that because that is going to make this facial animation look so much better I mean honestly it looks good right now the AR kit doesn't look bad it uses the same thing that we use for our live link here but man wouldn't that be crazy if they're able to do this with the actual entire uh meta-human facial system that being said that's all I got for y'all today what you all think about this new update from Nvidia Omniverse audio to face I think it is freaking awesome and I'm gonna be playing around with it a little bit more just to tweak the shapes and things like that because honestly we have so many more freaking Sliders in here now so yeah it's gonna take a minute I'll see you on the next one peace out

Channel: JSFILMZ

Views: 8,440

Keywords: NVIDIA Omniverse Audio2face to Unreal Engine 5.2 Metahuman Tutorial, audio2face metahuman, audio2face tutorial, audio2face unreal engine 5, audio2face livelink, audio2face live, nvidia omniverse audio2face tutorial, omniverse audio2face, omniverse audio2face tutorial, omniverse audio2face metahuman, omniverse audio2face metahuman tutorial, omniverse audio2face unreal, omniverse audio2face plugin, omniverse audio2face realtime, how to install nvidia omniverse audio2face, jsfilmz

Id: 6QV3U5PENPM

Length: 8min 39sec (519 seconds)

Published: Fri Jul 28 2023