How Does WebRTC Work? | Crash Course

hey what's going on everybody so welcome to this webrtc crash course in this video what we're going to do is break down what webrtc is we'll talk about how it functions and how all the details actually work here and then we're going to use what we learned to build out a peer-to-peer video chat application so this right here is actually a live demo i have two different tabs open and we're going to build out something like this but we're actually going to go a little bit beyond this and make our application a little bit more functional than this but if you want to test this out this will be linked in the video description so let's go ahead and get started i'll run through a couple of slides we'll do an introduction and then we'll start coding this up and i'll explain as we go okay so before we get started i just wanted to let you know that these slides right here will be linked to the video description if you want to access them along with a detailed article to go with this so that'll be all linked up you can go ahead and check that out once the video is posted and let's go ahead and get started so first of all what is webrtc so as i have it in the definition here webrtc is a set of javascript apis that allows us to establish a peer-to-peer connection between two browsers to exchange data such as audio and video all in real time so this is real-time communication so the thing that makes webrtc so special is the fact that the connection is between two browsers and the data that transmits between the browsers never actually reaches a server so it doesn't mean that we don't need a server involved at all but as this picture shows right here let's say we have two clients that data is transmitted directly once a connection is made and never has to reach that server so this makes webrtc ideal for exchanging audio and video because any latency that would be added by having to hit the server first would actually cause a slight delay so i know some of you are thinking if it's real-time communication this sounds a lot like web sockets so websockets and webrtc are similar in a lot of aspects so i want to break down what web sockets are and how that functions and then we'll make some comparisons on when you might want to use one over the other so websockets the connection is peer to server right here so we have a browser to server connection versus webrtc where the connection is browser two browsers so one is peer-to-peer one is peer to server so they both allow us to communicate in real time but with websockets let's say peer one right here wanted to send a chat message to peer two they would send the message that message would go to the server and then out to peer two now if peer two wants to respond they would send that message to the server and then out to pier one so there's a little bit of latency with web sockets and having to first go to the server but it happened so fast that half a second or maybe a second wouldn't really make much of a difference for something like a chat message or a notification but if we wanted to send audio and video data through websockets that latency would be very noticeable i'm sure we've all been on a call or seen a video where the two people are talking there's a little bit of latency and they're talking over each other so that creates a pretty big issue right there so this is where webrtc thrives so by not having to reach the server first the connection is directly between the two browsers and that video data can be exchanged or audio data can be exchanged much faster now with that webrtc uses udp as its protocol to transport the data and udp is very fast and more on that in a second so if webrtc is so fast why would we need websockets if they both give us real-time communication and one can transfer data faster for us it seems like webrtc is the optimal path here any more speed in our application always seems like a better thing here so there's some limitations to webrtc so let's talk about this and then when you might want to use one over the other so first of all webrtc uses udp and udp is not a reliable protocol for transferring important data so the way udp works is that it sends data really fast but it never validates if the data was received so because of that if we're sending something like an audio or video data if we lose a couple of frames from our video it's no big deal it'll catch up your video might look a little bit funny for a second but everything will turn out and it's not a big issue if we're sending something like a file to someone if we lose a couple of bytes of data from that file that entire file can be corrupted so because of that udp is not reliable you would not want to use it for sending important data now also with webrtc there is no built-in signaling so this means that you can't just use webrtc and make a connection webrtc actually leaves it up to us to make that initial connection and then once a connection is made then webrtc takes over so this is where websockets and webrtc actually go good together we would use a process called signaling with websockets we would send the information between the two peers the connections made and then webrtc takes over so webrtc has its limitations in that way so how does all of this work so i'm going to give you a quick example first and then we'll actually start breaking down the details of this but at a high level overview we have two peers and if they want to communicate we would have one peer initiate a connection and they would say hey you want to chat and they would have to send this message out to peer 2. so they would send out a message it doesn't matter how it's done this can be done through signaling they can send that message through an email a tweet it's irrelevant but that message would contain some information about this peer right here so they would have to send their network information to peer to now if peer two accepts this offer they would go ahead and send some information back to peer one the means of communication right now it's not relevant but the second that information is exchanged data can begin to flow and now the two peers do not need a server involved they can start transmitting data so this can be audio and video data and that connection is made so what exactly are we sending between the two peers and how are we sending this data so as i mentioned earlier that means of communication in the beginning is actually irrelevant that could be a tweet or an email but usually this is through a process called signaling that's a more practical way of putting this into an application and that can be through websockets or some kind of third-party signaling service it doesn't really matter so you'd get the two peers in some kind of room so they can at least know about each other then they would transmit the initial data now in that data there are two main components there's a session description protocol an sdp and some ice candidates so first of all both peers will need to exchange a session description protocol one in the form of an offer one in the form of an answer and a session description protocol is simply an object containing some kind of information about both peers usually like the codec address media type audio and video so just a bunch of information about your network and how to connect to you so both peers will need to know about each other's network now along with exchanging session description protocols between the two peers both peers will also exchange a series of ice candidates and an ice candidate is simply a public ip address and port that could be potentially used as an address to receive data so each peer will typically have multiple ice candidates that are gathered by making a series of requests to a stunt server and they are exchanged between the two peers so we'll actually get into this exchange process and we'll go more into detail about what's happening so exchanging session description protocols and ice candidates so right here we have two peers right here and we have a server and then we also have stun servers that we're going to connect to so we'll go through that process again the one that we went through in that high level overview and the first peer will say hey let's connect here's my session description protocol so it'll be sent to the server through a signaling process and sent to pier 2. now peer 2 will receive that sdp and they can accept it and they'll say hey sure here's my stp let's connect the second that's exchanged the two peers are connected but data can't start flowing yet they've simply made that connection so before data can start flowing between the two peers we still need to coordinate the discovery of our public ip addresses so that's because most devices nowadays sit behind firewalls and nat devices so to do this we're going to use a method called ice so this means we're going to make a series of requests to a stun server get our ice candidates and then transfer them between the two peers and a stun server is not something that you're going to have to worry about setting up it's really cheap and easy to maintain so there's a lot of publix done servers and in our example we'll use a google stunt server so let's go ahead and continue with this process so what's going to happen here is once the sdp offer and answer are exchanged this peer right here will go ahead and ask the stunt server hey what's my public ip address now the stunt server will reply and they'll send over a series of ice candidates to this peer and this peer will send over the ice candidates to pier 2. now peer 2 will do the same they're going to go ahead and request make a request to the stun server and say hey what's my ip address or public ip stunt server will respond and the ice candidates are exchanged now once we find or the network finds an optimal path to communicate through with these ice candidates data can start flowing between the two peers and that entire connection is complete now i wanted to talk about one more thing and that is trickling ice candidates so how you send over ice candidates is up to you but there's a few ways you can go about doing this and let's go ahead and take a look at this process all over again and i'll try to break this down so when we first create our stp offer we also make a series of requests to a stun server to generate our ice candidates now what we could do is create our stp offer wait till all our ice candidates are generated and then send over the stp offer and the ice candidates all together now more traditionally what you want to do is not have to wait for this process to complete you want to send over your stp offer and then trickle in your ice candidates as they're created so you'll notice that we sent over the stp offer and then we don't get back our ice candidates in a batch they're sent over individually so we would signal over each ice candidate over to our peer as they are generated so we would send over the offer send over an ice candidate and another and another and then that process will continue so that is called trickling ice and we're actually going to run through this so it's going to help us fix a little bit of a delay here so let's go ahead and jump to the next slide and i want to show you this in action before we start coding so i have a live demo here so this is actually a gif in that live demo and let's go ahead and click on this and go over how this exchange works between the stp offer an answer so let's go ahead and open this up make sure my camera is on so the gif is actually at the top of the link right here so here i am this is all live right now source code is all here and what i'm going to do here is open up two tabs side by side so i want to make a connection between two peers okay so we have pier one and pier two so if we go through these instructions what we're gonna do is first create an offer so this is what an offer looks like so i'm gonna create an offer and you see this object right here is just a bunch of gibberish about our network we're just going to go ahead and copy the entire offer and i'm going to paste this over to pier 2. so we're creating an offer from pier 1 pier one will initiate that offer pass it over to peer two and this is the more manual way of doing this you wouldn't actually do this in a real application i'm just trying to or i built the demo so you can see that exchange process so peer two will get this offer and then we're to create an answer so whenever we added an offer here from pier one let's go ahead and create an answer from pier 2. so now we have an answer and you can see the object one's an offer one's an answer let's go ahead and copy this entire answer and send it to peer two okay so we've exchanged those and once we've exchanged the offer and answer i'm gonna go ahead and add the answer to peer two and now we see a connection both peers are connected so we have peer one and peer two and then pier two and pier one right here okay so one thing you'll notice that there was no ice candidates exchange at this point and that's because we're not using signaling so by the time the ice candidates were generated they were actually added to the offer so i didn't need to actually generate those on my own those were created so i didn't have to actually send those over because we're not using signaling so that's the demo check this out before you get started and let's go ahead and get right into coding this we're going to start this project completely from scratch so at this point i have nothing set up other than an empty folder right here on my desktop so go ahead and create this folder and open it up in your text editor so i have the folder and i have vs code open and i open up the project file so we're going to create the boilerplate code for this and then we'll continue from there so if you want to go ahead and download the github repo the html and css parts going to be minimal but if you don't want to copy or if you don't want to write all of this out just copy and paste that in and then just continue from where where we start writing the javascript code so in that folder we're going to have three files so we'll have an index.html file we're going to have a main css file so minimal css and then main dot js so let's go into the html folder i'll create some html boilerplate here so i'll just go ahead and auto complete that if that doesn't work for you just go ahead and type this out we'll add in a title so web rtc and make sure you have your javascript and css linked up so in my case i want to put the javascript underneath the body tag let's go ahead and move that and the css is already linked up so main.css and main.js so inside of our template here we're going to start with a div for our videos here so we'll create a div and the id here will just be videos and this is going to contain the two videos for our users so we'll just call this videos and then inside here we're going to use the video tag and this video tag is going to have the class of video player this is for styling so create a video player and then we're going to give it the id of user one so we're going to have two video tags here and one will be user one and one will be user two let's go ahead and close that up right there so we'll have the closing tag also and then inside of this or inside of the tag let's just go ahead and do auto play and then we'll just do plays in line okay so once this tag is set up let's go ahead and copy and paste this these will be our two video tags this will be user two make sure you have everything exactly how i have it and now let's go ahead and style this so we can see it so we'll go into the main.css file and the first thing i'm going to do is go ahead and go to the body tag so we're obviously not focused on best css and html practices and let's just go ahead and give the body a width of 800 pixels and then we'll center that so i'll just do margin zero auto okay so that's the body at this point what i want to do is go ahead and make sure it's responsive so we'll just do at media screen and then we're going to do whoops at media screen and max width and this is going to be 800 pixels so at 800 pixels we just want to make sure that the body goes to 100 so we'll just copy that and we'll just do 100 i guess that wasn't too much code i could have just written that out but i want to make sure it's responsive okay so after that i want to go ahead and make sure that i can see the video tag so let's go ahead and grab the video tag so just do video i haven't done a tutorial in a while so i'm a little bit rusty so let's set the width to the video tag and we're going to set this to 100 percent and then we want to be able to see that before we actually add a video into it so let's just go ahead and do a border for the border we'll do 2px solid and we're going to use green yellow so i want to make it obvious and then we'll just set some kind of background color so we can see it and for this we'll just do rgb and then the values i wanted to use were 63 62 and 62. okay so we have the video tag styled and what i'm going to do next is go ahead and go to the videos themselves so that wrapper and i want to make sure these two are in line so we'll just do videos and that is an id okay so for this let's go ahead and do display grid and then for the width we'll just do grid template columns or for the width of each column we'll just set that to one fraction and one fraction so we'll take up the full width of that grid right there and then we'll just do a column gap make sure there's some space between them we'll just do one em and then the width will also set that to 100 percent and then padding 10 pixels okay so that's almost there for the styling we're not going to have too much more i think we're only going to style the text areas next but let's go ahead and open this up in live server so open this up here and here we go so for some reason i'm not seeing that border let's see i'm supposed to see border 2px okay i see what i did there so this should be 2px solid or 2 pixels and that should auto refresh okay so that's all we have we have the two video frames now what i'm going to do here is add in some text areas so the in the first demo that we're going to work on we're actually going to manually transfer the sdp offer an answer the way that i showed you in that demo so we're going to have those text areas down here so we're going to copy paste copy and paste it so let's create them for that so we'll go into our html file and inside of the html file the first thing i want to do is add in a button to create an offer so we'll just go ahead and use the button tag and for this we'll just give it the id of create offer okay so we'll just write create offer for the actual button text and then underneath that let's go ahead and just add in a label tag and this will just be stp offer so we just know which one it is and then we can copy and paste these two actually let's add in the text area first so we'll just do text area and for this let's just go ahead and add in the id of offer sdp so this will be the actual value this will this will be how we query it with javascript so offer sdp and for the placeholder i guess we don't need anything there let's just leave it like that i don't need any other values so let's go ahead and just copy all of this bring this down here and then this will be answer or create answer and we'll just update this value right here then stp answer and for this text area let's just go ahead and update this to answer and we're going to add in one more button here and this will be to actually add the answer so once we receive the answer from the from pier 2 we want to go ahead and add it so we'll just change this it's funny how i'm really rusty after not doing a tutorial for a while so typing is going to be a little bit slow okay so we have a button create an offer the text area button for create answer the area text area for the answer and then the add answer okay so let's add some css i want to make sure that it's visible so let's go ahead and add in text area and this is why i mentioned earlier you can skip ahead if you don't want to type all this out just copy and paste it because this has nothing to do with webrtc i just wanted to make sure to give context i know some people get thrown off if they can't see the context of how all this was put together okay so we have the text area let's go ahead and open this up here we go that looks good okay so two areas of buttons all have ids and that's it for the boilerplate code so let's go ahead and move on to the next step here so we want to jump into our main.js file so make sure it's linked up and go in here and the first thing i'm going to do is create two undefined variables for a local stream so local stream will camel case that and then we'll also do one for remote streams remote stream so when we get the audio and video feed from our users we're going to store that inside of these two streams right here so we have those set up and what i'm going to do here is create a function to actually initialize all of this so we'll use an arrow function so we'll just call this init arrow function we'll actually make this an async function so let's get that going and then let's actually call the function right away so this function will trigger the second a user loads the page okay so we have our init function and the first thing we want to do when we initialize it is go ahead and grab the user's audio and video and actually display it to the dom so let's go ahead and take that local stream variable right there and we're going to go ahead and call await so make sure you have async and a weight and we're going to go ahead and use the navigator method right here so we're going to call this method and in here we have a bunch of constraints that we can use and to start here we're just going to go ahead and set video to true that means the video will play and for the audio we could set it to true but i want to make sure that it's muted just so we don't have some kind of echo and we'll set that to false so this will go ahead and add in our video and audio stream to local stream now right after we do that let's go ahead and do document dot get element by id and let's go ahead and grab user one so this will be our stream and we'll just do dot source objects so we have a video tag it has the property of source object and let's go ahead and set that to local stream so we're going to save this and really quick if you want to have or if you have any questions about these methods and you want to do some research on them i would highly recommend looking up the documentation so let's go ahead and look this up let's just do web rtc and i'm going to link this up inside of the video description here so right here you're going to see a lot of information about the functions that we're using specifically the rtc peer function or pure connection function we're going to use this so this will tell you about all the details i'm not going to explain them like the documentation would i will explain what's happening but i won't go into that detail so i'd recommend looking those up so this is actually the ron function is supposed to be get user media and let's save that and let's go ahead and see the output so i have live server on i don't need to refresh so let's go ahead and check this out okay so we've successfully displayed our local screen to our dom here and we're actually able to turn our camera on so if this isn't working obviously make sure the camera is on whether you're on a laptop or you have some kind of external camera make sure that's ready to go but this should be your output if everything went successfully so with that we're also going to grab the remote stream and for this we're just going to go ahead and call new media stream so we're going to set some kind of media stream but we're actually not going to set the value yet so it's going to be empty and this is what's going to be what the this is going to be what the remote user will have so let's go ahead and set that to remote stream we're not going to see anything but we can set the value right away so change this to to remote stream and then change this to user two so we're grabbing these elements in the dom and we're replacing them okay so that's it for this part it shouldn't change anything in here so we don't see anything for user two we just have a media stream in there but you're not gonna see anything because there's no actual content there so let's go ahead and jump to actually making a peer connection so in step two let's go ahead and actually create an sdp offer so above our local stream and remote stream let's go ahead and create a peer connection object so we'll call this pure connection and this will be like the source of truth for uh basically any connection so when we're connecting to another peer there's gonna be one object that's going to represent all the information about this connection so our local streams all this information will be added here and this is what's going to allow us to connect so we'll create this pureconnection object and let's go ahead and create another function down here and we're going to call this create offer so remember the first part in this process is to create an offer so let's go ahead and do that let's make it an async function we'll make it an arrow function too so always add in the async where i'm adding it there's usually a reason for why i'm doing that okay so we create the function and what i want to do now is go ahead and actually create the peer connection object so we'll go ahead and grab this undefined value and we'll just do peer connection and this will be new rtc peer connection and there we go so that's going to create the peer connection object this is going to be the source of truth for that connection now what i'm going to do here is go ahead and set up my stun servers i told you that stunt servers are free to use in most cases there's a lot of public ones that we can use so we don't need to set up our own in this case we're going to use google's so we're just going to go ahead and set up our servers here and let's go ahead and create an object so just copy and paste what i'm doing here or just copy exactly what i'm doing here i won't go into the details of where i got these servers you can go ahead and google that up and there's going to be a lot of urls that will point you to this so for this we're going to go ahead and do ice servers and this is going to be a list so we can add in multiple urls here at this point we're just going to go ahead and actually i think that was supposed to be so it's a a list and then an object and we're going to set the urls value so this is some google stun servers that we can use right here so we're going to do stun and then we'll do colon stun one dot one dot google dot com and then the actual this is the port so 19302 okay so we're going to do that and then we're actually going to set up two urls at this point so let's go ahead and add the comma there and then this will be stun two it's done two and then this will be left at one right there.google.com19302 okay so this is the stun server that we're to use and these are the urls right here so let's go ahead and copy servers and let's throw this into our peer connection so we want to let it know which stunt servers to use this is how we're going to generate our ice candidates so that process and where we went back and got multiplied candidates this is where we're gonna get those from okay so we set that up here and what i'm gonna do at this point is i'm actually gonna take this remote stream value and i'm gonna put this inside of the create offer so i don't want to set the remote stream until we actually create a new offer so it's going to create a new peer connection that way so we there's a point where you can actually add multiple peer connections if you don't want to have multiple users we won't get into that in this video but this is why we're gonna create it on the offer here okay so we have that set up the next thing i want to do is go ahead and actually create the offer so i'll just do let offer and let's just use a weight here and for this we're going to do peer connection so we're going to access the peer connection that we just created the peer connection object and we'll just do dot create offer so it has a method called create offer this will generate an offer for us now once we create an offer let's go ahead and actually add this to the peer connection so we create an offer then we're going to go ahead and set the local description so we have a local and a remote description for every peer connection there's always two descriptions here and at this point we're going to take this session description protocol and we're going to pass that in here so it's going to be an offer we set that to our local description okay so we set the local description and what i want to do at this point is go ahead and actually see this offer so we're going to do document.getelementbyid and we're just going to grab the offer sdp so that's this value right here in the dom offer sdp this text area and let's just go ahead and do dot value and this needs to be a string so we're going to set this to json.stringify and we're going to throw in the offer so it'll be an object but we're going to stringify it throw it in and then later we're going to parse it okay so let's go ahead and save that and let's check this out so if i click create offer oh okay i've completely forgot to add an event handler so let's go back here so we have the create offer method and i just want to add an event handler so we can actually create that so just above the init function i guess we can do this after it so let's just do document dot get element by id call this create offer so the create offer button make sure it's called that create offer right here so when we click this button what we're going to do is add an event listener and we're going to listen for the click event and we're going to call the create offer function okay so let's check this out create an offer here we go so that's the offer stp right there so we stringified it and we threw it in there's actually no ice candidates there at this moment so you're going to notice this offer is actually going to get bigger once we start adding ice candidates so let's continue here and we got that first part set up here so our offer is there but it's very minimal so what we want to do next is add in a series of event listeners that listen for when we add our local stream to our peer connection and when our peer adds in their stream to that connection so we can actually respond to these events and get our audio and video feed and actually share it back and also generate ice candidates and update our sdp offers and answers so underneath this section right here let's go ahead and create some space just above our offer let's go ahead and add in our local stream so this local stream once we got that information we haven't done anything with it other than add it to the dom so let's go ahead and get that and add it to our peer connection remember a peer connection is the object that connects two peers there's always going to be a remote and a local peer so the first thing i'm going to do is go ahead and go to the local stream and we're going to get all the audio and video tracks from this stream so we're going to call get tracks and this is going to be an array i believe so we're just going to go ahead and call for each we're going to loop through each track and we're going to add them to our peer connection so let's go ahead and continue on with the function here and we'll go to our peer connection object so peer connection and we're just going to call dot add track and we'll add in the track that we're currently looping through and then the local stream so this adds this track or all the tracks from local stream to the peer connection object now on the other end whenever our peer calls this same method right here they're also going to call local stream get tracks because to them it's a local stream for us they're remote what we want to do is we want to listen for this event so anytime a track is added by our peer to this object right here we want to respond to this so what we're going to do is we're going to call pure connection dot on track so this is an event listener and we're going to make this an async function go ahead and call the event so what we're going to do is go ahead and respond to this and we're just going to call event dot streams and we're going to get the first index of this and we're going to call get tracks dot each so we're going to loop through and do the same exact thing so in this case we're just going to get the track of our remote pier so they just added it we want to get that track and we want to add this to remote stream so remember when we set up remote stream right here so let me actually remove some space here so we set up a remote stream but there are no tracks added to the stream so it's there it's in the video player but there are no tracks so whenever our peer adds those tracks we want to go ahead and call remote stream and we want to add the track so it can actually begin playing inside of that video tag so we'll just call add track and we just want to add a track here so here we'll also add in local stream but we don't want to do that here we just want to add the track itself so now we actually have a video and audio track in that stream or whatever the user added so once that's done we still need ice candidates so what we're going to do is immediately when create offer is made or when our offer is created we're going to make a series of requests to a stun server so this is going to happen automatically that's done right here because we set up which stunt servers we want to use and we added that in right here to that peer connection so right away stun the ice candidates will be generated so we're going to call peerconnection.on ice candidate so this event will be called each time an ice candidate is generated and returned to us from the stunt server so this event will be called multiple times we'll call this we'll make this async and at this point half the time i probably don't even need to make it an async function it's just habit to do that because i've left that out before and it's kind of bitten me there so what we're going to do is first check that we have a candidate so we're going to do event dot candidate and then once this is a candidate let's go ahead and grab our offer so we're gonna take this value right here and we're gonna paste this in right here so every single time we get a new candidate the offer will be updated but in the dom we also want to make sure this is updated because we we set it once and we don't update it from here so we want to update the offer every single time a new ice candidate is generated now here you can't use the offer variable so we're just going to do pure connection dot let's see what was this this was local description not set local description but local description like that okay autocomplete's not working for me so basically what happened is once we set our local description we can access the description by calling peer connection local description like that so every time it's updated we simply update the dom now if we go back and look at this pure connection looks like we have an issue let's go ahead and check that local stream get track is not a function i think it's supposed to be git tracks that needs to be plural okay so here we go okay so now when i click create offer look at how much bigger the offer is so all the ice candidates were added to the offer so now we can actually send this over to another peer so it's ready to be sent now we're not using that trickle ice method that i talked about so it's all generated at once right here and added now what i need to do is be able to open up another tab right here and then create an answer and also add in ice candidates to that answer sdp so for this let's go ahead and create a new function we'll create this function underneath create offer so let's just do create answer and let's just go ahead and make this an async function make it an arrow function and this function is going to look a lot like create offers so what we want to do here is start with copying and pasting some information so our peer will also have to create a peer connection object they will also need a remote stream just like we do so to us they're they're a remote stream but to them it's local so we're the remote stream now i guess pier 1 is so let's copy everything up to the on ice candidates event listener copy that and bring that down to create answer so we want all of this information inside of our create answer function now we're going to do something a little bit different inside of that create answer as far as how we set the local and remote description so at the bottom of the create answer function let's go ahead and actually get the offer so we'll just do let offer and we're going to get this offer from the dom here so we'll just do offer is going to be document dot get element by id and we're going to do offer sdp so remember we're going to paste it into that text area so we're going to query that offer and let's go ahead and do dot value and we're going to get the value of that offer so before we continue i also want to make sure that there is an offer so i don't want an answer generated if we don't have an offer so we're going to say if not offer let's go ahead and return and let's just add in some kind of alert and we'll just say retrieve offer from peer first dot dot okay so that'll just stop everything right there but if we do have an offer the next thing we want to do is go ahead and parse the offer so we'll just do offer is equal to json.parse so we want to get back that object because right now it's all a big string so we'll just go ahead and throw an offer we'll parse it and then what we want to do is go ahead and call await peer connection dot set remote description okay so we set a local description originally when we created the offer so we have a local description and we have a remote description so each peer connection will have a local and remote description so in this case the offer is now going to be the remote description because we're the recipient and where the peer on the receiving end now what we want to do next is actually go ahead and create an answer so we'll just do answer and for the answer we're just going to do await peer connection and peer connection also has a method called create answer okay so we're going to create an answer and once we create the answer let's go ahead and call wait peer connection wow my auto complete is not really working today so peer connection dot set local description so the local description will be the answer for pier 2. so we'll go ahead and throw that in we set the local description it's going to fire off and toggle the same functions that the first the first local description value did when we called it right here so when we set local description it's going to call ice candidates and so on and continue there okay so we set our local description and we set a remote description now what i want to do is go ahead and add this to the dom so let's go ahead and do document dot get element by id and we'll just call this or we'll get the answer stp and we'll just set the value so we'll do value and that's going to be json.stringify and we'll throw in the answer okay so let's go ahead and add an event listener so let's copy this create offer event let's change this to create answer and then call the create answer method bring this down here and there we go okay so let's go ahead and test this out so what we would normally do is go ahead and open up these two tabs side by side let's close out the documentation bring this in right here so here we have two peers so one peer will always be the one sending the offer one will always send the answer so we'll create an offer let's copy this and let's try to create an answer right away so if we do that retrieve an offer from the peer first so just to make sure everything's working good let's refresh it paste in the offer so here we see the type is offer now we should be able to create an answer but like always in a tutorial there's always going to be errors fail to execute set remote description let's see what did i do here okay so we create an answer and for some reason peer connection is not allowing us to set an answer set local description okay so here's the issue i never added in the offer to set the remote description so now we have a remote description that's the offer local description is the answer okay let's go ahead and give that a test one more time let's open this up right here open this one up side by side and now let's go ahead and create an offer copy that bring that in create an answer now we also have another sdp with all the ice candidates this type is the answer and we have one more part of this to complete so we want to copy the entire answer bring that in right here and then we need to add the answer but we don't have a function to handle that yet so basically what we want to do is set the remote description for this peers so we only have a local description here here we have a local and remote description okay so we'll go back here and what we're going to do is add in a function called add answer here so let's see is that what i wanted to do next yeah so that was the right thing so let's go ahead and do let and this will be add answer and let's make this an async function let's go ahead and complete this so we're going through this process slow so you can actually see how all of that data transfers so i hope i hope you get the point by now and what i'm trying to accomplish at least at this point and then we'll just move on to making this more automatic so in the add answer method let's go ahead and get the answer from the dom so we paste it over here so the answer is pasted over so we want to get this area from the dom or get the answer from the dom and then set the remote description so let's just go ahead and do document dot get element by id in this case let's just copy all of that so we'll get the answer and we'll just go ahead and grab the value so we want to get the value of it so if it's there we'll grab the value and we actually do have to add in a variable because we want to check if this answer even exists so let's copy this right here if offer exist bring that down here and in this case we want to make sure there's an answer in the dom and then we'll just say retrieve answer from here then if everything checks out let's go ahead and parse the answer so the answer is equal to this will be json.parse parse the answer and after this we have one more thing to do so the last thing we need to do is simply check if we don't have a pure connection we don't want to add in multiple peer connections so if we'll just do if not pure connection so we use the not operator right here so if we don't have a pure connection or we don't have a current remote description so current remote description then let's go ahead and continue so current remote description why does that look weird no three r's there okay so if you don't have a current remote description let's go ahead and do pure connection dot set remote description that's gonna be the answer and that should complete the process so last thing i need to do here is go ahead and add an event listener and we're gonna do add answer that's this button right here and then we're going to call the add answer function okay let's go ahead and test this so we'll breathe this in create an offer copy it bring it in here create an answer looks like they're overlapping a little bit okay so we'll grab this answer bring it into pier one add answer there we go so that was the entire transaction process we just completed the entire cycle to how we can transfer data between two peers and make a connection so let's go ahead and clean up this function a little bit let's clean some of this up we have a lot of information here so what i want to do here is actually create a new function called create pure connection so let's go ahead and make this underneath the init function so we'll do create pure connection make this an async function so a lot of the logic that we have in the create offer and answer function are actually is exactly the same so what i'm going to do here is go ahead and copy everything from c so we'll grab everything from pure connection down to offer so this is all the same inside of both of these functions so we'll copy this let's actually remove it now and let's throw this into pure connection and then let's take the pure connection function let's minimize it for now and let's throw this into create offer so we're just going to go ahead and call pure connection so it's going to go ahead and run through all of that then what i want to do is go ahead and call the peer connection function right here so let's actually delete all of this down to getting the offer and we'll paste that in right here so there's going to be a few issues that occur here so first of all inside of the peer connection function let's see down here we're updating the offer sdp and then it also needs to be the answer sdp so let's pass in the sdp type so it can be dynamic and let's change this to sdp type so the offer will update the offer the answer will update the answers so right here we'll just go ahead and throw in the sdp type so we'll do let's see this was offer sdp like that and then down in the create answer this will be answer stp you just want to make sure i'm calling that right okay yeah so we're just changing the value and that needs to be dynamic so that's just some of the issues that come up with that this will be answer sdp okay so that looks a lot cleaner the create offer function is nice and short now we have all the logic that's repeated inside of the create peer connection function and we did the same for create answer so let's go ahead and make sure all of this is still working so open up two tabs side by side let's create the offer bring it over create the answer copy that bring it over add it and everything's still working so we just cleaned up a little bit and it looks fine so let's go ahead and move on to the next step and what we're going to do is work on signaling this data over so we don't want to actually have users have to copy and paste or offer an answer pass this over it's not really a practical application so we want this information automatically signaled over whenever a user sends some kind of invite and they happen to both join that same room so they can actually communicate so in the slides earlier i talked about using something like websockets to send this information over and signal it or use a third-party service and in this case we're going to use a third-party service so we don't have to build out our own signaling server so for this we're going to use something called agora and agora has a whole suite of tools for real-time engagement and communication and specifically they have an sdk for real-time messaging so it's agora rtm what we're going to do is go to agora io create an account download their sdk and then just go ahead and configure a few things and then we'll just go ahead and signal this information over so with agora you don't need a you don't need a card on file you can actually register with that one and they have a pretty good free tier where you get to use it for quite a bit before you run out of those use minutes they actually calculate things by usage minutes and by active users so i wouldn't worry about going over that for this tutorial and you can actually use it beyond that too so let's go ahead and go to this website and then go ahead and create an account so i already have an account with agora i'm going to go ahead and log into my console so once you create an account you'll be in your console and you're specifically going to want to go to this project section so once you create an account you're going to need to create a project or some kind of app i believe they call it we're going to create an app and then that app is going to have some kind of app id and then that's what we're going to connect our sdk to so i've used it before in other videos so let's go ahead and go to create you want to create a new project give it some kind of name like webrtc whatever you want to call it for the use case let's just do social and i guess we can just do chat room it's kind of irrelevant at this point and for the authentication select testing mode app id this means that we don't have to use a token to actually authenticate our users specifically for this tutorial leave it here so we don't have to actually use a token now i already have an app that i'm about to use so in my case i'm not going to create a new app i'm going to hit cancel if you don't have an app go ahead and submit it make sure your settings are set like this and once you create an app you should see it in your dashboard so i've already created a few apps here so i'm just gonna use one that i already have so for now just go ahead and leave your agora console open we're gonna go back to the main console and just go ahead and download our sdk and then we'll get back to getting our app id so in the resources center you should be able to go ahead and go to downloads and we are going to specifically look for the web sdk so you should probably start at all platforms go ahead and find web and they have a voice sdk video real-time messaging this is the one that we want right here so go ahead and click this it should download a zip file for you let's go ahead and bring that out open up the zip file go into libs and let's just go ahead and grab this and bring it into our desktop so i'm going to close this out that's the file that we want to extract i'll just move all of this over here and let's just grab this file right here and bring this into our project folder so this is the project folder that we started with we'll bring that in here so now if i open up my text editor i should see that project file right here so agora rtm sdk and then the version number so let's go ahead and copy that file path so i just went ahead and copied that and we'll bring this into index.html and above our script tag right here for the main javascript file let's go ahead and add a new one let's just do script close this out and i don't know why it looks so funny for me right now and then for the source let's just do source equals and then we'll just paste in that name so agora rtm sdk and then the version number if you're doing this at a different point this version number might look different so make sure you're grabbing exactly what's in here and that the file path is set up the way it's supposed to so we'll go ahead and save that and now we have our agora sdk so at this point let's go ahead and go back into our console we'll open this back up have way too many tabs open right now so i can close out this downloads here and you can go back into the main console and let's go back to projects and go ahead and grab your project here so at this point the only thing we need is an app id so it's hidden right now i'm going to make sure this is blurred out so i can reuse this later let's go ahead and copy that so i should copy it to clipboard and we're just going to go ahead and bring in this app id into our project so inside of our main.js file the first thing i'm going to do is go ahead and set my app id so i'll bring this at the top we'll just say app underscore id so we'll bring that in i'll go ahead and make sure it's a string and i'll paste that in here and the next thing we want to do is actually configure our app here so we have our app id we have the sdk downloaded so let's go ahead and move on to the next part and that is to make sure each user that logs in has some kind of unique id so we'll create a uid and you can do this in multiple ways at this point what i'm going to do is just make sure we generate a random number and that we make sure this uid is a string so we'll just go ahead and generate that number stringify it so we'll do math.floor and then in here we'll just do math dot random so we'll create a random number and we'll make sure it's a larger number so that way we have a smaller chance of these ids being duplicated so there's obviously better ways of doing this to ensure that two users don't have the same id but for now this is gonna do so after that we also need a token so usually this is for authentication at this point because we set our authentication mechanism to app id only the app id is the only thing we'll need so we still want to set that token variable but that's going to be null so later on if you want to change this to actually authenticate with a token you can just update it here so after that we'll need a client object so this will be like the entire interface for our client connection it's kind of like the pure connection object right here so we're going to access everything through the client so this will be the client right now it's undefined but we're going to go ahead and initialize all of this so inside of the init function let's go ahead and create this client so we'll do client and this is going to be equal to a weight then we'll do agora rtm so we have access to agora rtm and all the methods because of the sdk that we downloaded so let's do create instance and that's going to be the function that we'll use so because of the sdk make sure it's in here that's why we have access to all of this so now we have access to the methods and inside of the instance we want to pass in the app id so go ahead and throw that in we have that as a string you can pass in the string right here if you wanted so go ahead and add it and we now have our client so once we get our client we want to log in this specific client so this will be that interface for this user so we'll just do a wait and then we can do client dot login and in here we want to pass in our uid so we want to log in as our client we also want to throw in the token which at this point is going to be null so we'll throw that in and then we can move on to the next step here so once a client is logged in what we'll want to do next is create or join a channel here so we can go to the client object and call create channel so what this method will do is we'll create a new channel if one doesn't exist and if one does exist it'll just go ahead and join that channel so there can be multiple channels so this is how you can have multiple chat rooms multiple peers talking to each other and at this point this id is going to be called main so we're only going to allow one channel but later on if you wanted to make this dynamic you can grab some kind of value from the url so a user can create a chat room some kind of id would be generated in that url and that would be the channel name so this is how you can have a bunch of people on the same website but inside of their own rooms so for this tutorial sake we're only going to call this main but this value would be dynamic so we're going to create or join that channel once we establish that we have that channel we're just going to call join so now we actually need to join it as the client here so we create the client login create or find the channel and then join that channel so the next thing i want to do here is actually listen for when other peers join that channel and we're going to console this out just so we can see how the real-time messaging works here so we're going to create a function called handle peer joined so anytime another peer joins we're going to make an async function we're going to pass in member id so we want to know which peer joined so we're going to get their uid so each peer will have their own unique id so we're gonna have a function that lets us know when someone joins we'll do console.log can't spell right now and we'll just say a new peer has joined this room and then we'll just console out their id so we'll do member id so we're just going to get that oops put that in the wrong area okay so that's going to be in the parameter right here so this is going to be the function that handles the event but we still need to listen for this event so right here just below channel join let's go ahead and add an event listener so we'll do channel dot on so we have these event listeners and we're going to listen for member that's a string remember join so whenever a member joins what do we want to do so we're going to call handle peer joint so we're going to call this function and this will respond so let's go ahead and save this and let's make sure everything's working so let's open up the website let's go ahead and close these out so by the way if you want to look up the agora documentation and understand everything that i'm doing here go back into your console and go to the documentation right here or the api reference and this will explain every single function here so you'll want to select your platform in this case we are on web you'll find real-time messaging and i'd recommend studying this also if you're using agora and just try to understand the details and read up on what we're doing so each function here is going to be explained so let's go in here and first of all let's open up the console inspect and if you're seeing this right here it means that everything is working agora is actually connected so what we'll want to do is actually refresh that and open up a new tab here so when a new peer joins this channel we want to console out a new peer has joined so let's go ahead and paste that into the url and here we go we can see this consoled out right here so i'll try to zoom in but in real time what happened is we just got a message from this peer right here that they just joined so this is how we can respond to these events so when this peer joins what do we want to do so we want to create an offer here and then send it over to this peer and then eventually this peer will send a message back so let's go back into our code and what we'll want to do from here is send out a message to the peer that just joined the channel so when they join we get an alert we have their member id and we want to send them a message with our sdp offer but before we send out that sdp offer let's just send a welcome message just to see how things flow and then we can actually add in the data that we want to send so let's access the client object and let's use the send message to peer method so this is a way to send a peer-to-peer message so what we'll want to do is throw in an object here throw in the text value and then the actual message and that's going to be hey so this is a peer-to-peer message so how do we know where to send this well we need to throw in the peer id as the next parameter and here we have the member id because we know who just joined the channel so this message right here is going to this peer so as they join they get a message and now we need to add an event handler to listen to this so let's go ahead and create a method called handle message from peer and we'll make this an async function and here we're going to pass in the message that was sent and then the member id so we also know where this message was sent from so that way we can send a message back and forth so here we go we have the handler let's go ahead and just console out the message we'll do console.log and we'll just say message and that'll be message dot text so we want to send the actual text value okay so we have the function that handles the event but now we want to actually add in the event handler so here we're going to access the client object and then use the on method and we're going to use the message from peer event handler from peer and then we'll just pass in handle message from pier so we listen for this event so when this message is sent it'll fire this off right here this will be the function that handles it and then our peers should see the message so let's go ahead and test this out so we'll open up two tabs side by side we'll close one out and we'll actually start them fresh actually so let's copy this open this in a new tab i'm going to need to do inspect element so i can see everything as it happens and what we should see is a message here that says a new peer has joined so this is the first pier in the room right here so when this peer joins we'll see the alert that appear has joined and here we should see a message from this peers so let's go ahead and just copy and paste that and here we go so a new peer has joined a room and here we have a message that says hey so this peer sent the message to this peer so now we actually want to send some kind of offer we want to send some real data so we can at least see how that process works so we'll go back here and let's continue so in handle pure joined let's go ahead and actually create and offer the second appear join so we want to create an offer and then send it so i'll comment this out for now we're going to use this in a second and update the data but let's just grab the create offer function so right now we only create the offer when we click on these buttons let's actually just comment that out we don't need that anymore and let's go back up here so when appear joins we'll just call create offer and we're going to send the message from the create offer function so that means that we still need the member id because we're going to call this again so let's just paste in member id into create offer let's copy this right here and remove it and let's scroll down to the peer offer so create offer or the create offer function not peer offer so when we create an offer we're going to add it to the dom right away so you're going to see it in that text area and then we're going to send out a message right away so we're going to call send message to peer it does require a member id or the peer that we're sending it to and when we call the create offer function we passed in the member id so we need to make sure to throw in that as a parameter here and this message will be sent now we don't want to just say hey we want to actually send the object here so the scp offer that we created right here now we need to send a string here so what we're going to do is go ahead and stringify the object that we're going to send so called json.stringify and instead of just sending the offer we're going to end up sending multiple types of messages later so we want to specify the message type so we'll go ahead and create some key value pairs and we'll just say type this is going to be an offer so we'll just do offer and then the actual offer value so just call offer here and then throw in the offer okay so we're going to send this object here so on the other end we need to parse this so let's go ahead and go back to handle message from pier let's take the original message and then just call json.parse and we're going to parse message dot text so we're just getting that value right away so let's change this to message and let's just see the offer being sent over so let's go ahead and open both of these up and right now it looks like there was an error so let me just try to refresh this and here we go so we see a new peer has joined a room and here we see the message and that is the offer right here so this peer has gotten the offer so what i want to do here because the offer gets created right away so if i refresh this this will be peer two and then if i refresh this this will now be peer too so it keeps changing when that handle method is actually called or handle peer method is called so i wanna make sure that this offer is actually added right here so let's go ahead and continue here and inside of the handle message from peer method let's go ahead and check the type of message so let's go ahead and call if and then we'll say if the message dot type if this is equal to offer let's go ahead and add the offer to the dom so we'll call document dot get element by id and we're just going to get the offer sdp dot value and let's just go ahead and stringify this so we need to stringify the offer to actually paste it into the dom dot stringify or into that text area and let's just go ahead and get message dot offer so we sent along the offer with the original message when we sent it so we're just getting that offer and we're just going to add it to the dom so let's go ahead and just see how all of this looks here so let's open up bold tabs here i'll actually create a new tab copy that and right now if i refresh it we have no offer because the offer gets created whenever a new peer joins so if i go ahead and add this in here when this peer joins the offer gets created and then the message gets sent and passed in right here so we've passed in the offer but before we continue with the answer we still need to send over the ice candidates so what's happening here is the offer is sent over immediately once it's created but the ice candidates take a minute to trickle in so before we didn't have to actually send over those ice candidates on their own because by the time we copied and pasted it they're already added to that offer and sent over with it so we didn't have to worry about that but now it's being sent so fast so what we need to do is go back to this on ice candidate event so create peer connection this gets called and then every single time we create an ice candidate we usually update that description in the dom so in that text area right here so in this same area what i want to do is also send a message and at this point it's going to be the offer candidate or the ice candidate so we'll go to create offer here let's grab this entire method so this is inside of create offer and let's just copy this bring this into create ice candidate paste it in and at this point the type is going to be candidate so it doesn't matter if it's a offer candidate or an answer candidate so we're just going to go ahead and pass in the candidate itself we'll change this value so the type is candidate and then the actual candidate and the candidate will be event dot candidate so there'll be multiple candidates sent over so each time it creates one it'll pass it in and there we go so with the offer we're also going to send over a nice candidate and now on this end when we receive the message actually i just realized for a peer connection we need to know where to send this message so we have the member id that means for pure connection we need to also pass in member id and let's go ahead and pass in the member id the member id everywhere we call peer connection so create offer let's go ahead and pass in that member id and then anywhere else where we have it actually we'll leave it alone for now we'll just use that later okay so we send over the ice candidates but now inside of the handle message method let's go ahead and check if this is a candidate so if the message type is a candidate let's go ahead and process it so we'll do if message dot type is equal to candidate let's go ahead and actually add this candidate so we haven't used this method yet but there's a method to add to actually add each individual candidate so we're just going to first check if we have a peer connection just in case there's any kind of delay so we'll do if we have a peer connection then let's go ahead and continue and we'll just get the peer connection dot add ice candidate and then this will just be the message dot candidate okay so we're sending over the ice candidates and we're passing them in so let's just go ahead and check the message type here so i just want to see how this works so we have the message type we should see an offer and then multiple candidates here so let's go ahead and inspect console let's refresh it okay so here we should see a candidate there's one message we have another candidate and then we have the original offer so it looks like there was two ice candidates that we sent over so we pass in the candidates and then we add them to the actual peer connection and that looks good so we passed in the offer and the candidates so let's go ahead and actually create an answer now so create answer will actually be triggered when the offer is sent over so we'll just call create answer and let's go ahead and let's see what we need to do with the actual create answer so we're also going to send a message from the create answer method so let's pass in the member id and then let's go down to create answer respond to this and send an answer back to our peers so let's go ahead and pass in member id and we're also going to need this member id inside of create peer connection also inside of the answer so we'll throw that in because we are sending a message out from here and when we create our answer the first thing we want to do from here is actually send out that answer sdp so let's copy this from create offer bring this down here and after we add the answer to the dom let's go ahead and send it back to peer one so client dot send message to peer this will be the type of answer so we want to be able to identify this then we'll actually get the answer and then we're gonna send over this answer right here and then we're sending that to that member id okay so we create an answer added to the dom send it over and right now if we go ahead and just take a look we should see the answer actually be created and sent over so let's go ahead and close this out we'll refresh it open up a new tab and let's go ahead and join so when we join we should see the offer get created it's sent over the answer is created and the answer needs to be sent over here so let's go ahead and handle that so inside of our message from peers so we're sending a message back to peer1 let's go ahead and copy this question right here so if message type is answer let's go ahead and get the answer sdp right here so we'll change that and then we'll get the message dot answer so we're getting that adding that to the dom and that should be it for this section we don't want to create an answer what i'm going to do is go ahead and just call add answer so we'll add the answer so that should allow us to connect should add in the answer to the dom and the ice candidates will send out automatically so we call create peer connection so we have an sdp type and automatically this will be sent out as a candidate so that's already taken care of so that means the peer two should also send out the ice candidates so let's go ahead and just see how all of this is working looks like it's already sort of functioning let's try to refresh this all right there we go so let's open up a new tab again this is all done automatically so right now we have no offer i know i'm being a little bit repetitive but i'm trying to just go through it so we send over the offer we create the answer then the offer ice candidates were also sent over we create an answer send it over it gets added to the dom we sent over the answer ice candidates and a connection was made so we've completed this process now there's one thing that i wanted to take care of here and that is sometimes if i refresh this too fast i know you probably should just stop users from doing that completely but sometimes the peer connection object i'm sorry i mean local stream sometimes this local stream right here is not created fast enough so what happens is when the peer gets the message sometimes we try to create the answer and go through this process before that local stream was actually created and then we're calling functions on an object that doesn't actually exist so what we're going to do is we're first going to go ahead and check if that local stream exists for some reason if it was delayed we're just going to say if not local stream then let's go ahead and just paste in local stream we're going to create it and then just add it to the dom so we're just going to ensure that this is created so that's it you probably won't see the issue right now um it kind of happens randomly but now if i do ctrl shift r i'm just going to refresh it and then if i refresh it a bunch of times it should work and then switch between the peers and everything looks good so that's it for this video i hope you learned a bunch i'm actually doing a video for traversing media on his channel and in there we're actually going to create a real project with webrtc so it's going to look really cool we're going to have controls things like muting your audio turning your camera on and off and actually joining custom rooms and talking to peers so it's gonna look really cool it's gonna be a fun project i'll link that up in the video description once that video is done so it should be maybe another week or two before i post that

Channel: Dennis Ivy

Views: 45,595

Keywords: Programming, Software Developer, Dennis Ivy, Dennis Ivanov

Id: 8I2axE6j204

Length: 70min 42sec (4242 seconds)

Published: Fri Apr 01 2022