How To Setup Agora Video And Voice SDK For Unity | Multiplayer XR Rig With Video Chat #2

what's going on everybody this is dilma and welcome back to my channel so today i'm gonna be doing video two of the agora videos where we're gonna be doing not only we're gonna be streaming a video from agora but i'm also going to position that video right above the extra rig which is going gonna be really cool because we're gonna be able to play with other people we're gonna be able to see their webcam as they're playing with the extra rig so let's jump into unity and start working on it alright guys so let me show you what we're gonna be doing today which is to extend or agora video series by adding a cool overlay of the video on this character so the way that it's gonna work is this is gonna be part of my multiplayer you know video series so if you didn't watch those make sure you watch those because this is gonna make a lot more sense but basically i have an xr rig and this is using the xor xr toolkit it has a main camera it has also an overlay video which is the one that we're going to be placing in here and it has a canvas right we're going to be using this canvas to place a raw image that i you know that we generated on the previous video and the way that it's going to work is i'm going to be tackling that between the xr avatar so this is going to be the mode that is going to show when we have basically the video shown on the canvas and when i say the canvas i'm talking about the main canvas on the scene that i show you where the video was rendering on the previous video and then if we want to tackle it to the xr avatar cube it basically going to it's kind of like creating a tv and then we're going to be ourselves are going to be rendering right on the face of the player which is actually going to make it pretty pretty cool so that's how this is going to work it's just going to be rendering into this guy but to do that there's going to have to be a lot of code changes that we need to well not a lot but some code changes i wanted to make to our player or network player so if we go into a network player there's gonna be a couple of things that we need to add in here the first one is going to be a serializable field and this is going to store the agora user id the reason for that is because for each player i need to know what player you know basically which webcam is associated with which player so that we can position it correctly and this was kind of tricky to figure it out but i'm glad that you know that we're going through these so i'm just going to call it the agora user id and then i'm just going to create a basically it's going to be a network variable right the server is the only one that is going to be able to access this so just remember that and then the other thing that i also need to do in here is going to be two different variables because we're going to be tackling basically the player head is it going to be the square head which is going to have the video or is it going to be just the normal face so the first one is going to be a private and then game object and i'm going to call it normal avatar we can call it normal avatar phase you know how whatever makes sense to you and then this one is going to be the video video here avatar and then this is not going to change we're still going to disable the client input if we're not currently the owner i already show you how that works on the previous video the other thing that i want to do is i want to start the agora video automatically so when we you know when we're starting so i'm just going to say agora video setup and then i'm going to get the instance i'm going to say star agora that way anytime a client joins and he's an honor basically it's going to it's going to start the video automatically now that we have that there's going to be a couple more things we need to do here so i'm just going to also use the update method in the update method i'm going to say okay if this is a client is not the owner then what i'm going to do is i'm going to be i'm going to be returning but if we are the owner what i'm going to do is i'm going to create a i'm going to do a new check and i'm going to say agora user id value equal equal to zero so the only time when this is going to be equal to zero if if is if we haven't joined right we haven't really logged in into into a quora yet what i need to do is i need to tell okay can you a server can you make sure that you update the the user id for this agora user so what i'm going to do is i'm going to have to create an rpc for that so it's going to say agor update agora and it's going to be user id server rpc which we're going to have to be implementing but the method i'm gonna be calling in if you remember from the previous video is the actual gate agora user id so this is gonna give us the agora user id and then the other thing that i'm gonna do here which we're also gonna have to implement is i'm going to say okay you know what what i want to do is i want to allow each client to have access to determining whether they want to display the their video on the canvas basically on a conference canvas think of it like like that or do i want to have the video be on my head right so what i'm going to do is i'm going to also change the the placement and to do that we're going to have to do a little bit more coding so i'm just going to say i'm going to be adding the letter o to do that so if i press the o and i'm a client i'm basically going to be you know calling the server the server i'm going to ask hey can i change can i make sure that i change the you know the position of the of the actual video to all the different clients based on my selection so i'm going to say toggle and it's going to be called overlay placement server rpc i'm going to have to be implementing these two so i'm just going to start with the with the first one which is going to be the simplest one so this one is going to be remember we need our attribute which is you know server rpc and then i'm gonna do public void and in this case we're gonna be passing in a user id so i'm just gonna say the user id that we're going to be passing is an unsigned int and it's going to be the new agora user id because we need to store these on the actual network player so i'm going to say agora user id of value which is my network variable equal the new agora user id so this is pretty straightforward if this is set to zero that means that this network player doesn't know anything about agora and then i'm gonna say hey server can you save that can you save the user that you just logged in from agora into a network variable the title player overlay it's going to be called from the toggle overlay placement server rpc and and this is going to have to call into the server and the server is going to have to call into the clients so the way that it's gonna work is we're gonna have to create another rpc here and this rpc is going to be this one right here and it's gonna say server rpc and i'm gonna say public void and it's gonna be basically just that i need to do another thing here so i'm going to do also a client rpc and then this one is going to be void and then i'm going to call it the place overlay on players client rpc and it's going to make a lot more sense as we work as we work on it so and the way that it's going to work is that the server is going to say okay well i know that i know that all my different clients want to change the position of the basically of the overlay well i'm gonna this the client the owner is gonna call into that then the owner is gonna say the server is gonna say well i know there was a change on the overlay for the client so i need to verify every single client that there was a change so to do that we're going to be calling into this one so server rpc is going to say okay i got to change and then that server is going to say okay clients there was a change so we're going to all of us are going to change our player overlay i know this is maybe a little confusing but trust me that it works really really well so i'm just going to say toggle player overlay and then this is where that the clients are actually going to be changing their overlay position which you know we're gonna have to implement so i'll go back to this and then kind of walk you through one more time once we finish the the implementation of the toggle player overlay so this one's gonna say void and then toggle player overlay so in the toggle play overlay this is where the change is actually going to happen i'm going to be getting the transform so i'm going to say transform and then i'm going to say canvas screen and remember this is the canvas that it's going to display the videos for everybody and this is actually located on a videos game object i'm going to get the transform i'm also going to be getting the canvas for the player because we want to place basically the videos also on the player's head you can say get component in children and then i'll say canvas because there should only be one canvas in this player [Applause] and then i'm gonna say okay i also need to get the players videos it's gonna say player and then canvas screen transform that fine and remember that we were using the agora user id so this is going to allow us to find out okay what is the actual video feed for this player so i'm going to say agora user id the value and this is where the magic happens because i store this information for this network player now i'm able to say okay which one is the current video that is rendering for this network player now i can say if player video is not equal to no then i know that the video was currently placed at the in the canvas screen which is under videos for everybody so then i know that i can do a toggle right i can say well i'm going to be changing the position of that video to the canvas player that way we can place it now on the head of the player which is where the magic happens i'm also going to be changing the you know where the normal avatar i'm going to set it to false i want to place the video here avatar equal to true and then the else is going to be player video we're also going to be saying this is equal to so in the case where we cannot find the video on the on the main canvas then i know that i want to toggle it back to the main canvas so i'm just going to say player video is going to be equal to canvas player that transform that fine i'm going to do exactly what i just did in here it's going to copy that and then paste that in here make sure that everything it is placed correctly and then i'm going to be changing the the actual transform that parent that we can so that we can place it back into the canvas screen and then i'm just going to go ahead and switch these two this one is going to be okay now we are on the normal phase avatar and it's going to be false and then the last thing that i need to do here is because of the the repositioning and everything i need to make sure that i am changing the local positions i'm going to set this one back to zero and i'm also going to be changing that local rotation it's gonna be quaternion and then zero comma zero comma negative 180 remember we need to rotate it negative 180 degrees and then equal to zero so this might sound like not much code but it was kind of tricky to figure it out so let me kind of give you a walkthrough again so the way that it's gonna work is okay so once we join as a network player we're gonna start agora right the video feed is going to be starting from agora it is going to be us joining in and once we do that we're going to be checking okay have we set the user id for agora if we haven't make sure we call into the server we pass in the agora user id and then this is going to allow us to on our keyboard for now we can change this to be using the trigger button on the on the oculus device but for now it's going to be using the o letter and then if we press the letter o we're going to ask basically the server to let all the different clients that i'm changing my selection so server is going to say okay you know what i'm going to be calling place overlay player client rpc and it's going to be executed on our clients and then each client is going gonna say okay what do i need to toggle my canvas location am i'm gonna be placing that on the main canvas or i'm gonna have the video feed showing right on my head so go back into prefabs and then xray multiplayer with video and this currently has a network player associated with it and also an agora user id this is gonna be the network variable that we created and also the normal avatar normal avatar i'm gonna associate it with the head that we're currently seeing right now and then the video head avatar which is going to be this one right here is going to be associated with the xr cube you can change the name of that if you like if you want to make it the same as the prefabs you can also make that work but honestly that's everything that i wanted to show you guys if you guys have any questions about these let me know in the comments and make sure you clone this from github and give it a start because you're going to be able to test it out and also help me in the long term by providing a star that is going to allow me to create a lot more code to the open source community thank you very much guys

Channel: Dilmer Valecillos

Views: 333

Keywords: unity3d, unity, unity technologies, games, game development, game dev, game engine, unity engine, agora unity, agora.io unity, unity video player, unity video capture, unity 3d, agora.io, agora.io video, agora.io tutorial, unity tutorial, unity lesson, how to use unity, gamedev, c#, unity game dev, unity mobile game, video chat, voice chat, unity 2020, unity 2022, agora video sdk for unity, agora voice sdk for unity, unity 2019, agora real time, agora tutorial

Id: F6p8vIyEVgs

Length: 12min 41sec (761 seconds)

Published: Wed Dec 15 2021