How To Setup Agora Video And Voice SDK For Unity | Video Chat #1

what's going on everybody this is dilma and welcome back to my channel so in today's video i'm happy to introduce you to agora who provides real-time engagement communication this means that we can do real-time video we can do real-time audio we can also do messaging and so on in today's use case i'm gonna use agora to basically do a very simple video chat in unity i'm gonna walk you through how to set it up in the agora portal what do we need to do to download the sdk how do we interact with it immunity and basically give you a demo that is going to be the starting point of creating an extra rig that is going to allow us to not only display the video on a canvas but also display a video right over our head when we're using an xlrx so let's jump into unity and i start working on it so the first thing that we're gonna do is we're going to be integrating agora into unity so i'm using the agora sdk and if you go into the unity asset store you're gonna be able to download it and by searching for agora there's going to be two different assets that you can download if you want to implement video chat which is what i'm going to do in this video you can download the agora video sdk but if you just want to do voice which is a stripped down version of the video as you can you can do that as well so you can click on it download it and then once you download it you're gonna have something like what i have right now which is an agora engine folder once you get it downloaded the other thing that i also want you to do is if you go into the agora that i owe so i'm currently in the console because i'm already logged in and create a new project it's going to ask you for the project name and also what use case you're building and there's two different ways that you can do the authentication you can do an app id plus a token which is obviously recommended for better security or you can do a test mode app id which you can use for testing i actually did this one and i generated a temporary token so i recommend that you do that once you do that and create a project it's going to give you something like what i have right now one project is called delmar xr vr and if you have multiple projects you can you know structure them here and organize them so the other thing that you can do is you can also create a temporary token and that's what i did and this is gonna be very critical information that you need you're gonna need an app id you're going to need a channel name and then also you can generate a temp token so once you have that then you're going to be landing you know in your project you're going to have everything that you need here the the things that i'm going to be implementing today is going to be two different scripts one of them is going to be very coupled to the agora sdk and then the other one is going to be more to you know drawing on the canvas basically interacting with ui there's going to be two different scripts that are going to be very beneficial when you integrate agora into your system so the first one that i'm going to do is i'm going to focus on the agora unity video and just so you know currently i have two different pro two different scripts in here that i'm going to be integrating there's already game objects that already created but they don't have any implementation so if i go and look at agora unity video it's a basically an empty mono behavior and also an agora video setup which is another empty mode of behavior which is using my singleton implementation which you can download by going into the repo that i'm going to be putting in the description okay and then basically it's just gonna be this agura video interacting with the agora unity video which is the one that is closer to the agora sdk so once you have that there's also right now what i have in here is just a very basic canvas and this canvas is going to allow me to basically grab the video that i get from agora and then place it in in the canvas so the way that it's going to work is i have a videos placeholder in here is basically a panel with a horizontal layout group and if i were to do this just as an example let's say that i wanted to create a raw image and the images that i'm going to be creating are going to be 60 by 50 just so that i can fit in perfectly in the gui right here and you can see that as soon as i do that they're going to be aligning in there so let's say that you have multiple videos because we may have multiple people joining our session so there's going to be video images that get added and then if people drop from the video basically they're going to be you know they're going to be removed because we're going to be implementing that in the actual implementation so enough of that let's go ahead and get going and going to go into the agora unity video and the way that i'm going to do this is i'm going to be implementing everything and then we're going to be going through the code and i'm going to explain it to you [Applause] all right guys so that's everything that we needed to do for coding on the agora unity video just to give you an overview the first thing that we do here is we get the irtc engine this is basically the wrapper to the agora sdk it has all the different methods that we're going to be able to you know to connect to and then call backs and things like that for the video stream the token that i have right now this is more of a temporary thing normally they recommend that you have a token you know kind of like an authentication system that generates a token and i'm also going to be putting the basically what they recommend that you do when you're implementing agora this is to determine what the last error is so that we don't keep you know displaying the same error over and over and over and also the local user id so that i can store which user id is the one that we are creating a video for and this is going to be very helpful when we implement the xray with the video overlay that i'm going to be doing on the next video so the first thing is super easy this is going to be okay we need to load the engine right we need to be able to load agora we're going to store our token here as an instance variable we want to make sure that you know the engine doesn't already exist because it already sticks there's no reason for us to do this again and then we're going to just call into the get engine and then by app id it's going to give it an instance and then during development i think it's important that we do debug and also warnings and you know and this can be changed based on the the environment it just basically gives you more logging depending on what filter you passed in and then if you need to join a session meaning that we want to just start getting or video you know a stream then the first thing that we're going to do is are we you know is the engine no if it's not we don't want to do anything because we haven't really initialized it and these are going to be all the callbacks that we need to basically tie to on join channel success is going to be us if we join the channel successfully this callback is going to basically is going to be executed if another user is joining our session this is the one that is going to get executed if the other user is going offline this is going to be executed and this is going to be important because we need to basically listen to okay are other users joining our other users joining if they're joining then we want to make sure that we create the appropriate raw image and then that raw image is the one that is going to render the video the same thing when they go offline if we if we want we want to make sure that we know when they go offline so that we can remove the raw image and then these are just going to be any warnings that the the actual engine might throw we can also handle errors in here and i also i'm going to show you the implementation of that and then the next thing that we'll do is we'll just going to enable video we're going to enable video servers and this is really important because we're going to be joining the channel by a key and this one is going to help you you know leave a channel so if we are connected to our current channel then we can call the leave channel meta disable all the observers and make sure that we're destroying the current raw image for the user which in this case is going to be me and then this one is important because if we're quitting the application we want to make sure we don't leave any garbage so we're basically calling the rtc engine destroy and also setting the instance to know enable video is going to be important when we're basically passing the video let's say that we close out of the app well we don't we temporarily you know stop the session the active session on the app and then we get back to want to make sure that we pause the video we don't keep streaming because that's gonna save us you know it's gonna it's gonna save us money long term so we're basically saying if we're not passing we're gonna be enabling the video if we're passing we're gonna be disabling the video so it allows you to toggle between those two and then this is gonna be us if we're joining and things were successful i'm going to be passing in well this is passed by the callback the channel name the user id elapsed time so the local the local user id i'm going to be storing it based on the user id that i got from the callback and this is just some login information it's going to get us the sdk version this method i'm going to show you what it does but it basically gives me a location where i'm going to be putting the video which contains a raw image and then i'm also going to be making an image video surface which is what's going to be rendering the video for agora the other things that are also important if we're basically doing a multiplayer video stream want to make sure we capture the on user join so if another user join we want to get also a child video location we also want to make a video surface so that we can place their video feed on the canvas and i also want to make sure that i am setting you know who is that video surface for well it's going to be for the user that i got on the callback i'm also going to be enabling the video surface and i'm also going to determine what kind of video surface it is which is going to be in this case a raw image when i actually well when the user goes offline we also need to handle that we want to make sure that we are also destroying the raw image that we created for that user so that's what i'm doing here if i call back returns you know the user id i'm going to i'm going to look for that user id basically game object and i'm gonna destroy it and then i also handle errors in here pretty simple stuff the game the game object give child video location is pretty simple it's going to give us the location of the video game object and then i'm going to determine do i already have the child video which basically is going to look it up by my user id getting the game object if it's no i'm going to create a new one and i'm going to attach it to the video basically the pairing is going to be the videos object i'm going to return the video location and that's how i basically get okay this is going to be where we're going to be putting the the video either is a video of myself the one that is joining or if another user is coming in and joining a session as well when i say joining a session i'm talking about multiple videos showing in the canvas one for your webcam and one for the other person's webcam so this this method is really important as well this is how i'm making a video surface the video surface think of it as the basically the wrapper that is going to allow you to render the video that agora is sending us so i'm going to tell it okay where and where am i going to be placing that video surface i'm going to be adding you know on the game object and i'm adding a raw image i am setting the local scale i'm also getting the rect transform because there's a couple changes that i want to make too i change the size to be 60 by 50 because that's based on the ui implementation that i did i'm also changing the local position and i'm also changing the local rotation this is really important that you change the local rotation because the video is not going to be positioned correctly so you always have to go negative 180 degrees to rotate it correctly and then i'm just adding the video surface which again it's a component that agora provides which you can see that is available in their sdk okay so now that we have these does this work out of the box well no we need to still implement who is going to be calling into these methods right so that's what i'm going to be doing next if we go into the agora video setup this is going to be basically the glue from my ui to agora so let me go ahead and implement this and then i'll show you how it works well this is everything that i needed to implement for the agora video setup so a couple things channel actions are gonna be the actions that we're going to do if we're joining or leaving i'm also have a reference to a bomb because i have that in the ui i joined channel ban i also have to specify the app id the channel name the token this is to determine if these settings have been currently set if they're set then we're good to go if they're not set we're not good to go and then also reference to the channel bound text and also the image because i'm changing the text and also the image color this is important for unity android because we're gonna have to request permissions for the microphone and also the camera so that's why this arraylist is set up that way and then on the awake method i just get a reference to the channel button text i also get a reference to the image i set this by default to green we can also just set it to white i think that's what i did before and then i also have the permissions that i'm going to need to the array list and then i also set this object to don't destroy a lot so that we can keep it alive if we change different scenes and then on the start method i'm just validating making sure the parameters are set i'm also change setting the callbacks for the on click event if we wanted to basically if we wanted to click on join it's going to start agora as long as the you know the action currently is set to join and then if if we do it again basically it's going to be a toggle we're going to be leaving and then the update method is just for debugging purposes we can hit the s key to start agora we can hit the l key to to basically leave agora this is going to be important for the next video because we're going to be getting the agora user id and i'm just calling the instance local user id the stargoda is going to verify that we have the settings ready remember we want to make sure that everything it's been set the app id and the channel name check permission is going to be calling into the permissions api and it's going to determine and this is something that is available on the unity engine android namespace is gonna check okay do i have the permission already if i don't i need to request it so that we can use basically we can use the microphone and the camera if we're using the oculus device for using an android phone and so on so once you do that we check for permissions we're going to be loading the the agora engine by passing the app id and the token we're going to be joining the channel and then i'm going to be just basically changing the state of our join channel bound to basically leave so that people can leave the session if they wanted to leave the session then on the other method the leave agora also pretty simple it's going to crawl into leave and i'm also changing the state of our channel button text we also need to control the life cycle of the basically the state of the agora is the case so if the application is paused then i want to make sure that that is reflected on the video basically we want to stop the video from rendering and then if we go back and resume we're going to be basically calling back into enable video and then basically enabling the video from actually rendering and then an application quit wanna make sure we clean it up we're gonna be calling into unload engine so now that we have that going let's go ahead and jump into unity and i'm going to be testing but i'm gonna have to disable the camera because otherwise i won't be able to show you all right guys so i got the camera disabled now and i'm gonna have to join a couple of things in here and associate so if we go into the agora video setup which is basically just an empty gan object with the agora video set up a script i need to associate my channel button in that channel button is going to be this is the one that we're going to be changing the state whether it's going to be joining whether it's going to be leaving the channel and then i also have my videos a container here which has a horizontal layout group which i show you at the beginning of the video so once you have that we're gonna have to specify the app id right so i'm gonna go back into agora here and i'm gonna go ahead and copy the app id then go back into unity paste that and then i also need the channel name so for the channel name we're going to do something like delmar xr and then go ahead and copy that generate temporary token and then it's going to go ahead and paste that here and then the token i'm also going to be pasting that here so we have the video rendering and if i get close here the remember the size that i set for the for the video in the make video surface if i go back into the agora container videos if you remember the idea i set the videos and then here's the id the user id that we got from the from the callback and then i also created a raw image so this is our image that we created the size delta that we were setting in the code is 60 by 50 and then these are some of the parameters remember we did negative 180 and for some reason it changed a little bit i think it's because of the parents how their size but this is basically how that works so let me go ahead and join this session from my other computer and show you that next video as well okay so i got the other computer set up i'm just gonna go ahead and join as well and you can see that as soon as i join there's two videos now rendering the first one is my main computer the one that we're just working on and then this is basically my laptop currently looking up on me and then you can also see that we can leave the channel so i'm going to go ahead and exit out on my other session and you can see that as soon as i do that we can see the callbacks and user join this is when the user joined from my other computer and also a user offline got executed and then the game object got destroyed as well so if i wanted to leave this as well i could do that on my own computer and i can leave and you can see that now i don't have any sessions if i wanted to rejoin we can hit start and see how the video now reloads so that's everything that i wanted to show you in this video i'm also going to do a second video where we're going to be doing basically kind of like an overlay and in that video we're going to be using the exa rig and the extra rig is basically going to toggle between these avatar and also if i were to remove the head there or also the avatar it's going to have kind of like a like a basically a flat you know a flat face and i'm going to be putting the the actual video a stream right on the avatar face so that when we if we want to tackle it let's say that we want to display on the canvas we can display it on the canvas but if i want to display it on each character we can do that as well so i'll see you guys on the next video

Channel: Dilmer Valecillos

Views: 623

Keywords: unity3d, unity, unity technologies, games, game development, game dev, game engine, unity engine, agora unity, agora.io unity, unity video player, unity video capture, unity 3d, agora.io, agora.io video, agora.io tutorial, unity tutorial, unity lesson, how to use unity, gamedev, c#, unity game dev, unity mobile game, video chat, voice chat, unity 2020, unity 2022, agora video sdk for unity, agora voice sdk for unity, unity 2019, agora real time, agora tutorial

Id: HbXP3sSx9vE

Length: 18min 21sec (1101 seconds)

Published: Wed Dec 15 2021