Mastering WebSockets With Go - An in-depth tutorial

foreign Percy today we will be talking about websockets what they are and how we can use them and how we can use them in go to build real-time applications if we think about it regular HTTP apis are dumb like really dumb we can fetch data by sending a request to the server if we keep fresh data on our say website we have to continuously request the same data over and over and over and this is called polling and this is like having a kid in the back seat asking are We There Yet instead of just having the driver say we are here now this is the way we started to develop websites long ago but it's kind of silly isn't it thankfully developers have solved this with Technologies such as web sockets webrtc grpc HTTP to stream server-side events you name it there's a bunch of bi-directional communication protocols out there right now we are going to look at web sockets because it's one of the oldest ways to communicate in a bi-directional way between the client and the server what this means is no longer is only the client able to send requests to the server about new data like if is anything new is anything new but we can have a bi-directional communication between the client and the server where both of them can send data to each other so the server can push data and the client can request data in this tutorial we will cover websockets what they are how they work and how we can use them in go to communicate between servers and clients we will explore some common pitfalls that I've seen in websocket apis and how we can solve them one of them such as authentication there's no support in websockets by default for authentication you have to solve that on a a higher level on the HTTP layer during this tutorial we will be building a chat application where you can enter different chat rooms and send messages and the web server will be written in go and the client will be written in simple HTML and JavaScript and the patterns we learned today can easily be applied and adapted to different programming languages so you don't have to focus just on the languages used you should be able to easily rewrite this using react or a go client friend so before we begin coding let's understand just quickly what websockets are so websockets are defined in RFC 645 if you want to go really deep you can look that up um but websockets uses regular HTTP to initialize the communication let's go ahead and look at a simple step-by-step basically it begins with the client sending a HTTP request to the server but this HTTP request will contain a special HTTP header which is the connection upgrade basically we the client tells the server that hey I'm using HTTP right now but I want to upgrade this connection and if the server is supporting websockets it will respond with an HTTP 101 switching protocols so basically using to communicate and the server will respond with hey sure let's do that and then a long and running websocket will be connected between the client and the server uh using TCP and it will be connected until either party sends a closed message so both the server and the client can close the connection whenever they they feel like they're done this is how you connect a websockets basically it's a really simple protocol and there's nothing more to it but you really want to dive deep into the internal workings you can look at the RFC so why do you why would we need web sockets I mean they are very common in chat applications when you have one client sending a message that's supposed to be distributed to all other clients so you would have a websocket between the server and all the clients and whenever somebody pushes a message it will get pushed to all the other clients websockets are also commonly used in games if you have a multiplayer game that is web-based for instance and you can then use websockets to push data to everyone who is connected currently we see websockets in real-time data applications where you have this live feed or whatever um and basically anytime you need data just quickly pushed out to the slides we will Begin by setting up a simple HTTP server that holds our web application using a file server and we will do all this in go and I will refrain from using any web framework for now just to make this tutorial smaller and simpler we will use vanilla JavaScript and hopefully you know some JavaScript and I can't expect you to be somewhat familiar with go but let's begin let's begin by making myself smaller and let's move myself up here fine right so let's begin by creating a module so creating my module and we're going to create a new file called main.go which will be running our server um we're going to start out without using any websockets we respond the whole bulk of the application to be up and running um so let's go ahead and do a package Main and we are having a main and we will call a function called setup API Now setup API will be responsible for setting up the HTTP routes that we have currently we will only have a file server so we're going to host the HTML and JavaScript stuff using a regular file server so let's make an import so what we have here we have setup API setup API will handle the base routes and it will host a file server and it will host the directory which is located in the relative path of called front end now we don't have that path yet so let's go ahead and create it we have front end and we don't have a front end so let's build that it's going to be a really simple uh HTML now let's I'm going to quickly fill this with some mock data dummy data um whatever and we're gonna have a HTML body and let's make some quick headers like a tile and let's call it websockets with programming Percy we don't need anything more uh let's go ahead and create a body now we're going to use HTML to render the information that we are using and we're going to use JavaScript to connect to the websocket let's go ahead and do a div class I'm going to add some CSS as well the reason why we are coding this live or is because I really don't like tutorials where they kind of go and oh you know here you have a bunch of code Jump Right In let's go hopefully I can show you guys how easy this is to set up so let's create a chat application some titles which shows currently we're showing a hard-coded chat room which is general which is the place where everybody will go now we're going to allow our clients to select which chat room they are a part of so we're gonna have a form and we're not going to have in any action because we are gonna trigger a websocket event whenever they change their chat room we're not gonna make the form send a post request and let's just leave a label so they can enter what chat room they want to be in and we're gonna have six and it's going to be an ID of chat room so we can easily select it and let's have it called chat room in name as well and let's break through and let's break the row and we're gonna have another which will be the submit button so they whenever they press this form they will actually change the chat room whenever they submit this form will trigger a event on the websockets which will change the user's current chat room boom there's no logic applied to this yet but we will add that very soon let's add a text area where the called message area and this is the location where we will show any data that has been sent on their websockets let's call it chat messages read only because we don't want them to modify this and we are naming it and and let's add some rows so we want to have this is just purely styling with uh four rows and let's just make 50 columns that's so it is 50 characters long and let's say some placeholder welcome to chat room and let's close it let's close the text area my bad like that and after the text area let's break the row and let's have a simple form again where they can enter messages so it will be chat room message for instance and we will have a label for um messages and let's just put it like that and we will get the input value so we will have a text input and let's call it message and let's name it message let's just break the row one so twice so we have a little bit of space and we're going to add submit button and the value will be send message so we have a we have a few headers showing them that this is the chat room we have a form where they can select the chat room to be in we are showing the chat messages and we have a a form where the users can send any message that I want so let's go ahead and add a script tag which is where we will maintain our JavaScript [Music] I just want to be perfectly clear that this is not how I recommend you to structure your web applications this is just to show you how we can easily get started with websockets let's have a default chat room selected and we're gonna have a function called change chat room which will allow the users to change the chat room and we're gonna do document get element by ID and I think we call it chat room we did call yes so input value we're going to grab is called chat room and let's just add some checks so if it isn't new and if new chat dot value is um selected chat because if they are already inside of the chat room we don't want them to switch chat room you should only be able to switch to a new chat room and we're going to return false now we are returning false because we're going to call this function from the form later and if we don't return false the form will try to navigate to a URL but we don't want to navigate away from this website so we are always returning false returning false will make the submit button not try to navigate so we can change chat room we also need to be able to send messages so we have exactly the same thing basically let's just copy that function because we're going to do the same thing but instead we're going to grab the message ID and I guess we can do new message copy that and we don't need to check if we're in the same chat room so whenever there's a new message we want to print that now whenever somebody visits our websites we want them to connect using their websockets so let's make um let's make the let's enable that but let's not connect yet we have nothing to connect to but let's make sure that we are ready to connect whenever we have the server up and running so whenever the window on load is called which is when they open up our websites we are going to grab the chat room selection and we're going to apply on submit so whenever they are pressing the submit button on the form we are going to call Exchange chat room function so we have a function which print the new chat room that is selected and whenever they press the submit button on the chat room selection form we will trigger this function that's what we are doing here we're applying a listener to the on submit function and we're going to do the same for chatroom messages but instead of change chat room we're going to call the send message so now we have that in place we are going to do one final piece before we start fixing the server and all the browsers have the window object and to check if the browser supports websockets we're going to see if the websocket is window websocket is available and let's just make a console log for now this is the place where we will connect very soon connect to web sockets but we can't connect right now so let's just make sure browser does not support web sockets right we have the JavaScript in place so we can send messages we can change chat rooms uh let's add some super fast styling just because we want this to look not good but just slightly better than what we have we're going to overflow I won't cover what I'm doing with CSS this is I can't expect you to understand somewhat what we're doing let's make it fill the whole to try speeding through this so that we are actually have a color predefined so let's use that it's kind of a gray cool stuff and let's whenever we're whenever we're in the center we're gonna Auto margin we're gonna have a width of 50 this is so our little chat screen will cover half slight border so we see where we are and that's right so we have a super simple website that will display the data we have a super simple HTTP server so let's go ahead and run it and see that we are actually doing the correct thing so right right my bad nothing is happening because we actually need to start listen and serve and we're going to listen on port 8080 and we're going to return nil whenever there's an issue let's see now we start it and we can visit localhost 8080 and you should be seeing and we visit the local and if we visit localhost 8080 now we should be seeing this we have the chat room where we can select we can we can see messages in the future and we can send messages here by pressing this button and feeling something great we have the foundation up let's see if we can start using web sockets now so let's begin by connecting our front end to our back end using a websockets now in JavaScript it's super easy there's a built-in websocket library that you can use without importing every anything and it comes built in with the browsers so we can create a new client by saying new websockets and it accepts a URL and the URL is going to be prefixed with the protocol to use uh let us open it up to show you guys so if we go down here where we say we're going to connect to the websockets let's actually do we can do new web sockets and it accepts the URL now the URL is going to be localhost in our case but we need to prefix it with the protocol and I need to mention that there are two protocols when using websockets there is the WF which stands for web socket and there is the WSS which stands for websockets secure basically it's using https or HTTP so WS is http 2fs for secure is https but we need um certificate to host this and I won't cover that until the end of the tutorial but I recommend you to use W secure mode whenever it's possible so we're going to connect a new websocket to a local host and let's do documents Dot location host plus slash WS so this will Connect using the websocket protocol to a localhost 8080 slash double slash websockets now the reason I use slash websocket is because we will be hosting a slash websocket API endpoint in the server which is used to do the connection this is kind of normal you usually see Slash WS when there's a websocket involved and let's store the connection in a variable which we can use later now if we go ahead and open up the front end and open the developer View and we should if we visit the website we should see there's a problem with applying the Handler on submits so there's an issue on chatroom selection go up and see all right my bad I typed in the wrong name in this in the form ID it should be chat room not chat ROM um so if you go ahead and save that and we reopen we should now see no errors but we do see an error that we can't connect the web sockets and if we open the network tab you can go here and see the HTTP headers and you can see the connection hopefully you can see the connection let me zoom in you can see here the connection upgrade header that has been sent to the server and indicating that we want to use a and websockets sadly our backend does not yet support websockets so we will get this error however let's go ahead and upgrade the back end so it actually can accept this request now in the back end I will be adding a manager.gov the manager will be used to manage anything related to the websocket and I'm going to add it in the main package and we're going to use gorilla uh gorilla is a gorilla is a very common uh developer who have a lot of libraries out there and they have a websocket library which is super common which I do recommend and let's go ahead and go get it so go get GitHub go get github.com Gorilla slash websockets and we have that the first one we're going to do in our manager is we're going to create something called a websocket upgrader and it's part of their uh the web socket Library now the websocket upgrader is used to take an HTTP request and upgrade it upgrade it into a websocket connection instead of regular HTTP requests so you're going to see how smooth this is in a second let me when it's not a function so let me upgrade that to that and I'm going to set up read buffer size and a write buffer size um this is just to make sure that our clients doesn't send these huge huge huge package packages we will cover more on that later so let's create a manager struct and we're going to create a Factory function for new manager which returns a pointer to a manager and right now the manager won't have any data it will just be an empty structure but the manager will have a Serv websocket function which is a HTTP Handler so we need to we need to have the HTTP Handler signature which is accept the response writer and a request a second parameter and we're going to log that we have a new connection and we're going to upgrade the request so remember whenever this function will be called it will be when we receive a request from the client to upgrade the regular HTTP connection so upgrade regular HTTP connection into websocket and to do this we use the websocket upgrader that we created before and we use the upgrade function we take the response writer we take the request and we can leave the header as nil for now let's just check that we let's just print the errors for now this is not a real application let's not handle we don't want to focus too much on that for now we're not doing anything let's close the connection whenever we're done um so the manager now has the possibility to accept websocket connections we just need to make sure that our API also actually handles this gracefully so in this setup API I will create a new manager calling the new manager function and then right below here let's add the endpoints and let's do handle Funk remember it's at slash websockets which is what the front end will expect and we are calling the manager serve websockets Handler so whenever we get a new request now at slash websocket the manager will take that request and upgrade it into a websocket this is fairly fairly simple I believe so let's let's go ahead and run it go run start.go because we have multiple goaling files and and as you see when we visit the website we no longer have an error if we go to the network tab we can let's open this up and let's make this a little bit bigger so you guys can see it let's zoom in and you can see that the request was sent for their websocket API it's a regular HTTP get method and we see that the response we got was 101 switching protocols and hopefully you remember the websocket standard as we discussed before we send a connection upgrade and we receive a 101 that we're now switching a protocol you can see here in the developer tool that we have this messages field and here we will be able to see any messages that are sent over the websocket in the future that's good to know so the websocket request that was sent go to that go to messages and you should be able to see the requests that are being sent over the websocket that is very good to know in the future right so we are now we actually have a server and client who are connected by websockets and this is all it took however we are not doing very much with it for now so we are going to upgrade this a bit of course so one of the first things that we want to fix is that right now we are handling all the logic inside of the serve websocket Handler now I like separating uh stuff so we already have a manager but I'm gonna go ahead and create a new file and call it client.go and client.go will be handling everything related to a single client so whenever we have a new person connecting it will create a client in the back end which we can use to manage it so let's go ahead and create the client struct and each client is half a one-on-one relationship with the websocket connection so say we have a websocket connection per client and also I like to have a pointer to the manager who manages the clients and we will see more on that later and again let's create a factory for the clients and each client will accept a web socket connection and a manager which we can reference and we will return a pointer to our client s so let's turn a client and connection [Music] and the manager will be the manager inserted but the reason for this is because we from the clients we'll do some stuff to direct it to the manager like broadcasting to other clients and stuff and this way we have that opportunity so let's also make a Client List which is a type and the type is a map holding the clients as keys and just a bull to make sure they are there now this is what we need in the client and we also need to update the managers so the manager can maintain these clients so let's jump back to the manager and in the manager we will update it so that we have a client which is a client list so that's a map of clients and since we can have many people concurrently connecting to the API we need to make sure that we protect this by a rewrite mutex let's also update the the factory function to create The Client List whenever we create a manager so we don't get a nil pointer exception now we will inside the manager also add some helper functions for adding and removing clients to the manager so let's go ahead and say that whenever we have a in the serve websocket what we want to do is we want to accept it we want to upgrade it and whenever we we do that we actually want to remove the con we don't want to close the connection right away but we want to create a new client so we're going to create a new client and we're going to pass in the connection that we just upgraded and the manager which is ourselves for now and then we want to add this client to the manager so that's what we're going to build but it's going to be m add client and we're going to pass in the clients so let's go ahead and build that function right away so pointer reference and add clients we accept a client as input and we return nothing because there's nothing to return so let's go ahead and lock the manager oh let's see manager my music so we're going to lock the manager so that no when we have two people connecting at the same time we won't modify the map at the same time and have a collision and whenever we're done we're going to unlock the manager so let's check if this client is already connected and we can do that what's he doing name okay so in here the manager Client List let's name it clients instead makes more sense so let's check if the client is part of the map and if he is we're going to actually delete we're actually going to oh sorry my bad we're not going to delete him um whenever a new client is added we're going to add him to the map and just add a Bool says he's connected now this is concurrently safe and we're fine but let's also the thing I was doing remove clients so again we accept the client as inputs we lock and we defer on unlock so if okay we're going to check if the client exists and if the client exists we're going to close the connection and we're going to delete him from the clients list so what we have here is we have an ad client which just simply whenever we have a new connection we add them to the manager and we can also remove them which we will Implement later but it will remove the client from the client lists and this way we have a little bit more nicer structure when we keep improving what the client can do we won't have this massive blob of code inside the serve websockets so now that we have our clients in place we can start implementing some Logics for our clients such as reading and writing messages and this is pretty this is a pretty easy task reading and writing messages on the websocket is a really easy task but there are a few things that people usually miss out and you won't notice them at first because it will work but one of those um mistakes is the websocket connection from the gorilla package actually only allows one concurrent writer at a time and if we do write messages straight to the connection it will probably work unless we have a lot of traffic but if somebody spams or if we have a ton of clients standing on the connection or we will have a problem but there's an easy fix to this we can use an unbuffered channel to prevent the connection from getting too many rides at the same we will cover this really soon but let's go ahead and open up the manager and inside here inside the serve websocket once we have added the client we will actually start two Google routines per clients one will be to read messages and one will be to write messages uh and let's go ahead and do go client dot read messages we will very shortly jump into the client and implement the read messages also so whenever a client connects we add it to The Client List and we also start a process which reads messages so let's go ahead and take a look at read message um the connection has a read message function so let's just create the function and we're going to have a client read messages and in here we're going to have a loop that runs forever and inside this Loop we will check for messages and the client connection has a read message and if we take a look at it you can see it outputs three types the message type the data payload and an error now the message type let's go ahead and say message type payload and error message type is defined in the RFC there's a few different message types you have ping on data binary message and you can read more about them in the RFC if you're interested usually you only use like a few of them either binary or text Data now let's go ahead and first check if it returns any error so you might wonder when it will return an error and it will return an error whenever the connection is closed or is unexpectedly closed for some reason so let's go ahead and make sure that we actually first off we want to break the forever Loop if the connection is closed so but let's also go ahead in the websockets we have a a few helper functions to check this one the error one of them is is unexpected close error which will be thrown if we try to read from a connection with which has been closed so we're going to check if it if the error we got is either websockets close going away and we will also check if it's websockets close abnormal and if it's any of these errors so if the connection is closed or abnormal closed for some reason we want to log that and in this application I will simply print it to the standard out but you might want to handle that in a real application now let me make this clear what's going on we get an error from the read message whenever something is wrong or the connection is closed but if this is a regular close if the connection is closed in a normal fashion the client send a close and the connection closed we don't want to log that maybe because that's not really an error that's why we check for the abnormal Clauses so whenever we get to this log statement the connection has closed without the client or server sending a close message something has gone wrong so we want to log it otherwise we will simply break now whenever that happens we want to remove the client from the manager so I'm going to put a defer function up here because this break will close the for Loop and the the go routine here will actually end it's so it will trigger this one so clean up connection here which we will do by going to the manager and removing the clients now remember remove client will close the connection remove it from The Client List so we actually have a nice solution to clean up clients who are exiting or having network issues or whatever let's just go ahead and print out the message type for now and let's print out the payload for now and let's print the payload as a string because it's a byte array so we want to read it so this this is how we read messages we call the read message it will return an error if the connection is closed so we need to make sure that the close is normal and only unless it is normal we want to log it because then we have an actual error and if the connection is closed we want to trigger the function to remove the clients from the back end so we're cleaning up anything great so we can actually test this really quickly so we can now read messages so let's go back to the front end and let's go up to the send message right now we're only logging the message on the console but let's go ahead and set and send their websocket message now sending messages in JavaScript is really easy remember we have the connection assigned here and we can actually easily in the send message here to connection Dot send because we want to send a message and we can do new message value so we will take whatever the user has inserted into the form and send it on the connection as easy as that nothing else so let's go ahead and restart our golang program so the server has booted up let's open up the UI once more and this time let's skip this let's clear the log and let's restart the application so we have a websocket connection up let's go ahead and type by a message here hello and let's send that message and you can see here that the message was sent on the web socket you can see the payload here hello so let's go back to the back end and we can see two things being printed we have one which is the message type so this is a message type of one and you could probably go here to see the message types we can probably find them declared inside of here so in gorilla you can see the message type defined this is from the RFC this is not related to the gorilla this is all so you can see here we print the one because the message type is a one so it's a text message it could be a binary message it could be a close message ping or pog so these are the message types that we can expect now we are not going to do anything particular with this but it's useful if your server accepts different kinds of messages like if you accept binary or text messages you can kind of have a switch based on the message type we can also see the payload being printed hello so this is really nice our server can now accept messages read them and print them have come a long way now do you remember that I said that the connection could only write one message at a time and this will be a problem if we're doing concurrent stuff which we are so we'll actually need to update this and gorilla themselves have an example of this and the way they recommend you to solve it is using an unbuffered Channel which blocks any concurrent rights to the connection so whenever a client tries to send a message we won't do client.connection.write message because that would allow one client to spam 100 messages and they would be concurrently written we don't want that instead whenever a client writes a message we want to take that message and write it to our unbuffered Channel which we then read from this way we can control so that we always only write one message at a time I hope that makes sense it's sort of like a and gay you know you can you can the client can spam messages but they won't go directly to the connection but they will be taken one by one from the channel instead let's go ahead and update the client the client want an address which is used to avoid concurrent rights on the websocket connection that's what we want to achieve so let's create a aggress which is a channel which we write by file arrays too let's go down to the factory function and make sure that we create this channel whenever we create a new client so it's time to go to the manager the same way we did with read messages we want to do go client dot rewrite messages so whenever we get a new client we want to start a process which writes messages I suppose this could be inside ad clients but let's keep it up here now for now let's go to clients let's go down let's make a new function which is write messages and this is going to do the same thing so whenever we have something break we're going to go to the manager and remove the client so this this defer is here to help us clean up any unused clients or clients that are having issues and we're going to have a for loop again which runs forever but this time we're going to have a select and some of you are probably going to be wondering why I'm doing four selects right now instead of just a full range on the loop but we're going to add more stuff to the select later so bear with me that's why we have a flick here so we're going to accept and we're going to check messages and okay from whatever the aggress channel half so we're going to read all the messages from the egress and we're going to use the the payload and okay if the bull mentioning that egress is fine and still up so the first thing we want to do is check if the channel still up because if okay is false that means the aggressive channel has closed for some reason and if we have closed let's go ahead and be nice guys and have our websocket connection write a message to the client so the server will write a message here because we have we are having issues with our egress and the reason for this can be multiple stuff probably we remove the client or something went wrong we have to notify that the the other side of the connection that this connection has to be closed so we're going to send a close message and the payload that we're sending is going to be nil the second parameter is the payload it's going to be nil so let's do a check if error is nil and if we have trouble sending we probably have closed the connection so we try to send a message to the client that we are closing the connection but if we can the connection has probably been closed we're going to return the return is going to break the for Loop which will trigger the cleanup but if we were successful and we could receive a message we're going to try and go to the connection and write the message so websocket we're going to send a text message we're going to send the payload as it is and we're going to check if we can send it and fail to send message and let's change this to our printf and let's log that the message was sent so what we have here in short terms instead of having each client writing directly on the connection which we're not allowed to do because it can't handle concurrent rights instead when messages are sent we will write them to the Ingress Channel and the egos channel will one by one select the messages and actually fire them away on the websockets this way we have a concurrently safe solution which is really the way to go so now that we have this in place and we're listening on the egress we actually have to have something right to the egress and that is something we can solve for now by making sort of a loop there's no process right now that writes messages to the egress but we can make a quick hack to test it and see that everything is working as expected we will make a small hack right now and the hack will be that every time we read a message we will send it to all the other clients we'll broadcast each message and so let's have a for Loop and we will Loop all the websocket clients and we will go to the manager and The Client List and for each client and each egress Channel we will write the payload this is just a hack for now to make sure that the address is really working we're not writing any unit tests that's not what we're learning here today so whenever I restart now whenever a client is reading a message it should also broadcast these payloads to all the other clients and those will be read and written to the those clients in a concurrent way so before we try this let's update the front end so that the front end also can receive the payloads that are written using the right message and before we do that we need to discuss a little bit about the events that websockets has so the websocket has a few events which you need to learn and they are close and the close event fires whenever the websocket closes you see error whenever there's an error there's a message whenever the websocket receives a message and there's an open which will trigger whenever the websocket is open so I like many times you can have like whenever the websocket connection closes maybe you want to reconnect if the websocket is vital for your application or you want to show the user some kind of error like we don't have a connection right now and but what we are interested in is the on message and we right now we only want to try if the right message is actually working as expected let's go ahead and do connection on message because we are listening on the we are listening on the message event so we are accepting an event and we're going to print the events and that's it that's what we need to change on the front end so we can go ahead and restart the back end and we can actually try this now by opening up and hearing the refreshing the website we can go here and we can send a message tests so we can see that the message test was sent and we can see that the message was actually received here so we're receiving the message that we sent ourselves and that's because right now inside the client read message when every message that goes through the websocket will be sent to all other clients this is probably not how you want it but we can verify that our solution actually works so whenever we send a message we get the message event and it's being printed so I hope that makes sense like the whole reading and writing messages it's not really from a code perspective it's not really hard what makes it a bit confusing is the whole we is the aggress channel but just try to try to think that we don't want to write a million messages at the same time the connection can't handle that we want one message at a time so instead we have this pipeline in front of it which we write to which handles the processing now this is great we can connect between Defiance and server we can send and receive messages and this is all great and we basically have a basic setup for a websocket API up and running now one thing I kind of like to do is having some sort of events or type system in place that makes scaling the implementation a lot easier what this means is basically that I like implementing some sort of RPC system or basic format that is sent on the websocket because as you know right now we can send any kind of text message there's no control on how that text message should look there's no typing at all and basically the if we have one kind of message in as in this case we have a chat which sends sends receives and outputs to all other clients that's fine that's easy and it works but what if we have two different things we want to do say change chat room now the way you do this or can do this or have multiple other scalable things happening is that we can kind of wrap each payload inside of a event class and basically each message that is sent is wrapped inside the event and we can use um we can have a type field on the event which tells us what to do with the message so we can use that to Route the message let's go ahead and take a look let's go to the front end first and let's go up at the beginning here of the script tag let's go ahead and add a new class called event this is what we will be sending and receiving through the websocket and this will allow us to have more control of whatever users are sending and so we can know what to do with it and we will want a type and a payload this is very similar to how websockets work you send the message type or binary type or whatever and the page so that the receiver knows how to handle it basically we're doing this but our payloads will be wrapped in a similar fashion so whenever we create a new event let's make the type into type and the payload we insert will become the payload now this is what we will send and receive so whenever we receive an event we need to somehow manage them let's create a function called route events which accepts an event as inputs and let's see if the event type is undefined we can't do anything if we don't know what type so let's just alert for now no type field in there then that's an error we want to just alert that let's add a switch and we will use the type field to switch and how you implement this is basically up to you I'm just suggesting that you can do something like this to have a more scalable approach to your websockets communication let's go ahead and say we have a new message event and this will trigger whenever a new message is sent on the websocket so what we will do is that we will for now just log it new message and we will break and the default what should the default be that's alert unsupported message and let's break so we have our event type we have a route function which accepts the events and triggers a certain feature based on the event side now let's add one more a nice little helper which will be send event and send event will accept the event name and the payload this is just a wrapper so we can send different events from the front end in a unified way so let's say we will create a new event and we will insert the event name which is a type and the payload and then we will do connection dot send and we will always want to send the events as Json so this little helper feature will take the event name take the payload and Json it and send it away on the connection so once we have this we can go down to send message which is no longer going to send on The Connection by itself but instead we're going to call the send event function send event and we're going to say send message event and let's put in new message dot value as payload now what we have is the ability to send messages but the messages will be parse or formatted as an event type and let's also go ahead inside here whenever we receive a message we don't want to Simply print it but we will expect events to come back from the websocket as well we send it in the event format we also expect it in the event format let's go ahead and do event data equals and we will Json parse the actual data from the event so the payload that is received on the websocket will be parsed and we will store it as that now we know that it should be formatted as our event class so let's go ahead and just simply assign it object.assign new event and we will want to assign the Json data and now we have an event object we want to route that event so instead of printing the messages whenever we receive them we will parse them with Json we will create our event object from them and we will route that event object so this should lead whenever we receive new message should be printed to the console great now hopefully that works as we expected to however the backend has no idea about the events so we kind of need to reapply this in the backend we want to do the same thing in the back end depending on the event type we want to Route it and we want also to make sure all outgoing messages are formatted in this event format so I'm going to go ahead and create a new file and I'm going to call it event.go and this will maintain all our event related stuff so let's have an events track and let's make sure we have the Json type type and we also have the payload we don't want to Marshal the payload we want to leave it as it is so I'm gonna say there is a Json raw message and we called it payload so this structure here is the same as the class that we're sending from the front end now the reason why we leave this as a raw message is because the user should be able to put whatever payload they want and and this way we won't Marshall it and that will allow them to send Json blobs in whatever way they want and so our different events can kind of expect different formats internally so that's really nice when a message is received in the backend we will use the type field same as in the front end to Route it to the appropriate event handler now an event handler is a function which will perform some kind of action based on the event that is received so let's go ahead and create the event handler and it's a function signature which accepts an event and the client to Center events and outputs an error now this is how our functions will look that can be applied through the websocket API let's go ahead and implement the first one which we have already created in the front end so we create a so let's create the event send message which is given that we will send whenever a new message is sent from the client and let's take a look at the Json raw message that I was explaining so different kind of event handlers will probably want different kinds of payload so the payload for sending a message maybe not be the same as changing the chat room that we want to be in so we can modify these and have different kinds depending on the event handler that is triggered in our case whenever a new message is sent let's create a new send message event and we will want to send the actual message that the user is sending let's say that that is encoded in message and we want to know from who it is so let's add that so whenever uh send message event is triggered we will expect a Json blob with the message and the from fields to be the payload now we have the events on the front end we have the events on the backend what we need to do now is we need to make sure that the manager knows how to route these events and I like to store it in the manager you can you could store it in the client as well it depends a little bit how you structured your application but I love having it in the manager because usually in the manager I have database repositories and stuff and I can do things from there so we need a way to store these handlers then you can have it as functions and a simple switch statement but a switch statement can become really really really long if you have many handlers so it's not really nice I try to avoid switch statements whenever I can now we can actually do this so in the manager let's go ahead and add a new field it's going to be the handlers which will be our event handlers so let's make it a map with a string as key and the idea here is that the type will be used as the key and allow us to grab the event handler so go ahead and modify the factory as well this time let's do it like this we want to store the manager and whenever we create the manager we want to also create the handlers let's make a map string event handler and let's call a function let's call it setup event handlers and we're going to return it and we need to create that function setup event handlers and basically right now we only have one event and it's the send message event so M handlers events and message is going to be a function and that's that function is going to be and the send function right now let's just make it print whatever we receive so let's create a function that actually fulfills the event handler signature so we're going to have a funk it's going to be send message let's just name it whatever for now and we will accept an event we will accept a client and we will trigger an error and we will print a print the event when it is received and return a nil so let's assign that function to the event handler so whenever we look for this event we will find this function and it will trigger a print line now we're not entirely done we need to Route them we need to check the map somewhere so the same way we did in the front end let's have a route events function it's going to be part of the manager this time routes events and the input will be the first parameter will be the event and the second will be the client and we will output an error so we're going to do a check in the manager's handlers if the event type is present and if it is we're going to execute the Handler with the events and the clients I hope that makes sense see you guys and let's return nil if not and if we don't find the event let's have an else statement let's return ER or dot new and there is no such event type so what's happening is that first we check if the event type is part of the handlers which is a map which uses the event type as key so whenever we receive a message that has the type set to send message we will trigger send message and that will print and let it handle here because we search for it and if we find it the function is stored in the Handler and then we execute Handler basically by passing in events and the clients and so we are almost ready to test this we have one final piece that we need to implement before we can test it and that's the read and write messages in the clients they need to respect that we are now using events instead so we're going to make a small adjustment firstly the event address is going to accept the events it's not going to say raw bytes anymore it's going to accept events let's change that and once we do we need to modify the read message so in the read messages we will remove all of this and inside here we will create a event structure so remember we are receiving a message and we can remove the message type right now we're only sending text types so what we are doing is we are expecting the payload that is received to be in the event format so let's go ahead and Marshal the payload that we have received into our newly created event and let's if we don't get it let's just say error marshalling events and let's just print it for now and let's break maybe you shouldn't break here I mean if they send one message wrong maybe it's a bit harsh to close the connection because if we break here you will clean up but I mean maybe log it whatever in this case let's break now here we have an event which means we can actually go to the manager and do manager route event and we can route the event and the client is from and let's check if there's any errors and let's just print it so now the server expects events to be parsed and whenever they parsed we are routing them and allowing the manager to handle them also we need to do the same thing in the right messages but instead here we're receiving them so let's Marshall instead let's Marshall the message so we get the Json data and let's print errors I love how we print all the errors don't do that in a real application handle layers somehow and let's see we should replace the message here with data and that's everything now we can go ahead and restart and we're going to test this out so basically what we have changed is that we send and receive events which looks like this they have a type they have a payload I mean you can add more stuff here and the payload is basically generic or not generic but the payload is handled by the event handler function so it's up to the function they will just receive the raw Json it's up to the function that receives it to handle it so we have done this for the backend and the front end so let's go ahead open up the front end again you can clear everything we can reload and if we go here now and do a message and we send that message and we can check it you can see here the message isn't it or test if I send tests but it's actually formatted as an event we have our type which is send message and we have our payload which is test in this case remember the payload can be a Json object because we're expecting a raw message and we can see from the back end we are receiving the send messages and since we are receiving them here in the read message we are marshalling them and then we are routing them and Route event does have a send message Handler because we added that in the setup event handlers so here we say whenever we receive send message trigger this function and this function will simply print the events great so before we fix the event handlers to do what they are meant to do because fixing that is a pure goal based thing it's not so much about websockets that's more just brushing it up the whole format and how to set event websockets up is basically done just a few things that you have to fix to make it really nice and there's security and heartbeats and we're going to start with heartbeat so websockets allows both the server and the client to send a pin frame and it's a special kind of message that is just used to check if the other side of the connection is still alive and that's why it's called a heartbeat so you can sick like hey are you still awake and if they don't respond you can kind of assume that that connection has dropped and you can leave not only do we check if another collection is alive but we can also keep connections alive so our websocket that is idle for too long remember we're still kind of relying on the HTTP protocol here as we're wrapped inside of it so if the keep alive is triggered because nothing has happened it will shut down so we will use ping pongs to kind of keep idle connections alive for however long we want because if you're having a chef app maybe you don't want to kick people after idling for one minute or so so whenever a ping is sent we want the other party to respond with a Pog and if no response is sent we will assume the connection is no longer alive so to implement this we actually only need to touch the server code because browsers by default using the websockets inbuilt the inbuilt websockets will actually respond to Ping messages by default so you don't have to do anything on the front end that is handled automatically but we need to fix things on the back end so let's go ahead and go into clients because this is client related and let's define a few variables that we will use and one of them is the pong weight and palm weight is the duration for how long we will await the path so if I send a ping I Will Wait a maximum of 10 seconds before I drop the connection and we will create a second variable which is the Ping interval which is pound weights times nine he divided by 10. Tongan Ting interval is how often we will send things to the client note that this has to be lower than the Palm weight if we have a pinion interval that sends slower than the pound weights the Palm weights will always cancel for instance if we send a ping each 15 seconds but only allow the server to wait 55 seconds between the parks the connection will be closed because there will be a 10 second Gap and so always have a ping interval that is lower than the pound weight uh also we can't multiply by o o we can't multiply by 90 basically this algorithm calculates 90 of this value so um that's what we do so we allow the server to have a 90 time frame to wait between the pings so we need to update the server to send the pings to the clients we will do this inside the client and we will do this inside of the right messages so inside write messages we will create a ticker so Time new ticker and it will tick depending on the Ping interval and this is why we have a select case here because at the bottom we will have a new thing to listen for and it will be let's see so I go through right place sorry we should be here so in case we receive a tick from the ticker we will do a log print line ping and we will send a ping to the clients so the pinging is handled inside the client quality so let's go ahead and do connection right message websocket ping there's a div there's a remember this is a separate type and it has to be the Ping message otherwise the front-end won't know how to handle the message and we will send that MP by array we don't have anything to send so if this happens and if error not equals nil let's see oh sorry we need to do that and let's just write message error in case something goes wrong and we will return to cancel the go routine now this simple change will make the clients send ping events to the front end and it will automatically return a pawn and we and the reason for this is because RFC tells us that the Ping and pong messages should trigger automatically now the browsers that all support websockets today they do this automatically so you should have no issue unless you use a third-party websocket implementation then it can change um for the server is sending things to the client the client response with a pong um but what now uh what we can do is we can handle a we need to configure a pong Handler so we are sending pings and we're receiving punks back but we're not doing anything with the pumps so whenever we receive a Pog we want to update the timers to reset because we have told the server or we will tell the server that we are waiting 10 seconds between each pog so we actually need to handle that somehow we need to tell the code that we have received a pong and we will reset the timer so let's go to read message and inside read message before we start the for Loop forever we will configure a wait time how how long to wait for the pump let's go ahead and do if error connection set read deadline and set read deadline is a function from the gorilla package which allows to set a time for how long we should wait so let's go ahead and do time.now and we will add pong weight so we will take the current time we will add 10 seconds to that and that is how long we will wait for the pong message to be received so let's just print if this goes wrong this shouldn't really fail ever I think because we're only setting a timer here now we're setting the timeline for how long to wait for the pong message when we start reading messages for the clients we also need to update a Handler luckily for us we don't need to add a custom event for that but gorilla have a function called set pong Handler which we can use to apply a certain Handler that is used when the pawn is triggered so whenever we receive a Pog message it will trigger the function that we assign here so let's do client dot let's call it pong Handler and then let's go down to the bottom and let's create that function so in the client we will have a pong Handler and we're receiving a pong message string error this is how the prong Handler should look so let's print it print line let's just print pong we have received a pong and we need to reset the timer remember because if we don't reset the timer the timer will run out and close the connection so let's return connection set read deadline the same function that we started here so we start the timer here and we wait for a pong message and when that pong message is received we reset it so let's go ahead and do the same thing here so we are using the current time we're adding the Palm weight limits great now we have pinks and we have punks that will keep our connections alive you can restart this the back end and connect a client so let's open the front end let's reconnect and let's go back and now when 10 past 10 seconds has passed we should see a ping pong and as you can see here we're receiving pings and we're receiving Parts I can't stress how important it is that in the pong Handler you actually reset the timer I've seen many people miss this and it's like why is my connection is dying reset the timer and you're good to go so implementing a ping pong heartbeat is actually really easy using the gorilla mask so another thing we need to fix when we're regarding security is that one rule in Security is to always expect malice malicious usage you can do evil stuff they will and let's go ahead and Implement a really easy fix for something called jumbo frames so let's expect people to send really really big messages and we want to avoid that we can actually do that by setting a limit whenever we are reading a message in the back end so the gorilla websockets package has a set read limit which will set the maximum size in bytes for how large the message that is received can be and again this is something you need to be a bit cautious with and you should really know how long your messages can be if you have a chat and you allow people to send messages that are more than I mean unlimited inside this is probably not a good idea but if you know how big your messages are going to be generally or if you have a limit on how large they can send you you can calculate the bite size and use that to set a hard limit and this is all you need to do inside the read you need to set the limits and if we start up and go to the front end and if we restart the connection and go here and if we just you know let's go ahead and copy that and copy copy copy copy copy copy send a jumbo frame we're going to see that this was sent let's go to the back end and obviously bam it's just my computer is a little slow but we closed the connection because the connection tried to send a massive amount of data and they shouldn't be allowed to and if we try to send a new connection we will see that it's closed so we have fixed the so we have fixed the jumbo frames let's go ahead and fix another simple thing that is really important and right now anybody can connect from any domain so we're going to apply a little course stuff because we all love course so inside the manager we're going to create a function which checks the origin of the connection and allow us to set that we only want to allow connections from certain regions and this is to avoid cross-site request forgery so it's really important that you actually do this and to do this we're going to create a new function and we're going to call it check origin and we're going to accept a HTTP request and return a booth this is the function signature that the websocket upgrader will expect so um if we return true we will allow the connection if we return fault we will dismiss the connection so let's go ahead and grab the origin from the request and it's part of the header so let's grab the origin header and we're going to make a switch origin and in a real application you should probably make the regions like configurable from an environment variable or whatever but for me for now I'm just going to allow anything from port 8080 localhost will be allowed and we're going to set the default that will return false so this function will allow anything from localhost 8080 which is where I currently hold the API so this function we need to apply it so scroll up to the websocket upgrader and the websocket upgrader actually has a field called check origin as you see here it expects a function which have this signature and that's exactly what we have created so let's assign it and we can restart the API and everything should work as expected still when we visit so let's go ahead and make a new connection first let's go back the messages sent let's close that let's try changing that so I changed the ports to 8081 instead let's go ahead and retry it so whenever we connect from a domain that isn't you can see here the stock this was finished we were not allowed to connect and you can even see the logs request origin is not allowed so this is good to limit from where your clients can connect and it's really important that you do so one other really important aspect of any API is that you sometimes is that you sometimes only want to allow connections that are authenticated for instance now websockets doesn't come with any authentication utility built in but we can solve that pretty easily since web sockets are built on top of HTTP what we will do is we will Implement a solution that authenticates the user before they connect to the websockets and there's a few common ways of doing this some time ago or a few years ago websockets allowed basic authentication inside the URL so you would do like my name is Percy my password and the websocket URL and they thought it would automatically handle that for you however that has been deprecated and is no longer valid so we need a new solution there's two solutions I've seen out there which I kind of both like but I prefer one of them so the first one that is widely used out there is that you have a regular HTTP endpoint that you can authenticate against uh simple slash login for instance and once you're authenticated you will get back a one-time password or a ticket that allows you to connect to the websockets this ticket can be added to the URL as a get parameter now a second approach is that you allow users to connect the websockets and then you allow them to send a authenticate event or payload that allows them to authenticate and verify that they are allowed to connect I prefer option number one where we have this ticketing system because we don't want uh like spammy Bots to connect and drain resources when they shouldn't be allowed to visit the websock so let's Implement option number one and in our case the flow will be a user Connect using HTTP they get back an ODP or one time password and they send this back to the websocket endpoint in the URL and if it's valid they get to connect and this will all take place in the serve websocket function um so let's go ahead and start updating the front end enough talk yeah let's go to index.html and we will update a few things first we want to notify the users that they actually are connected so let's add a connection header and we will say connected to websocket false hard-coded messages are the best and let's go ahead and add a simple form down here so that the users can actually connect let's make it a little let's have a border three pixels solid black and we're going to have a margin to the form above right and we will have a form and let's call it login form and inside the form we will allow users to insert the username and this could be implemented into your current Authentication pretty easily probably um we're building it from scratch now but in your application if you already have an application and you already have that in place you can probably just allow that endpoint to return the authentication ticket for the one-time password back so let's go ahead and allow a label for passwords and it will read password this would be a really simple login you shouldn't do this login in let me get that set and we will do password and the name will be let's go password that's the input and let's break some rows and let's we could just use the margin but I'm lazy let's have a submit button and the value is login so that's our simple form that will allow users to log in now we still haven't updated the backend to accept plugins but we will do that shortly but before we do that we need to update the connection because right now we try to connect whenever they load the website so we need to fix that so I'm going to add a few uh functions actually the first first function I'm going to add is the login function which will handle logins for us and I will also get the form and assign login form it's called and unsubmit we want to trigger login so whenever they submit we are triggering the login function now the login function will simply extract the data from the form so we will have username and we will get element by ID you could probably grab the form and then um grab the form data from that instead I'm doing this because it's a little bit quicker now in our uh application there's better way to grabbing the form data that I just want to say that then whenever we have the form data we want to send a fetch to slash login right and we want to so fetch will send our posts to slash logging for us with the username and password so let's go ahead and do a post and we want the body to be the Json format of our form beta and let's do so then if you're unfamiliar with JavaScript What's Happening Here is fetch will send the request but it won't wait for the request it will return a promise so whenever we say then we will wait for that promise to actually finish and then we will grab the response let's check if the response is okay and if it is we will return the Json data that is returned from the endpoint so this data here will be a Json object and it will contain our one-time password however we don't care about that in this function we only want to return the data and if fails for some reason we want to throw on on off on authorized and then when that is done we want to grab the data that is returned but the data here will be the response from this right here so the data will be the Json object and here we want to so at this at this point of time we are uh send authenticated so here we want to connect the websockets and we know that the data return will contain a one-time password in the OTP field so let's go ahead and call connect websockets with the OTP as input now this function doesn't exist yet we can to Clarity we can do this function connects websocket OTP we will cover this soon so we will call this function which will trigger the website connection for us along with the one time pack and just for my peace of mind if every anything fails we want to catch the failures and we want to trigger an alert and we want to return false because we don't want that to switch website whenever the logins so let's see okay this is it the login function in front end will send the form data to our endpoint we will Marshall the response of Json and we will trigger connect websockets when we are authenticated that seems about right so let's fill the connect websocket function what we will do is actually take everything from the window on load because we don't want to connect in the load function anymore and window unload will simply be assigning our event listeners so let's copy paste that part into the connect website we need to modify it a bit because at the end of the URL we want to append the OTP get parameter so websocket doesn't allow you to pass data it doesn't allow you to pass anything else other than the protocol and a URL but the URL can contain get parameters so that's what we're doing here we're adding that this is also the reason why I don't want you to send like jvt tokens and stuff in the URL instead we use a one-time password to make it a little bit more secure so so at this point of time we are basically doing everything that we need to we can just modify a few bits we need to update the the header whenever we connect so let's go ahead and use the on open on open will trigger whenever the websocket is connected so what we will do whenever a websocket is connected we will do grab the element by ID and we will grab the connection header and the connection header we will just simply update the HTML to connect it to websockets True something like that I suppose and we also want to do the counterpart which we can do whenever the websocket is closed so on close we just want to update that to false simple as that so unopen triggers whenever the connection connection is opened and on close will trigger whenever the connection is closed usually and let's unless it's a manual close I can recommend you to have some sort of automatic reconnection in the enclosed for instance if the connection dies due to the network dropping for the user or whatever for a few seconds you want it to connect unless they manually close the connection you don't want to re-authenticate okay so the front end should basically be good to go right now you could try it out you could start the application and you could visit the websites and we should the form is here so username and password and we can go ahead and fill something and we can log in and you can see here we're not connected to the websocket because our requests should have failed let's see do we see that let's go ahead this is not so responsive let's go ahead and trigger a login and we should see oh so here it is get element by ID not element my ID so once we've changed the typo and we log in we should see so I made the same typo here so if we change that save that go to the website and try to log in we will get a pop-up thing we're unauthorized because the request to slash login have failed but let me know the UI is doing what we expected it's showing the websocket connection and it's not connecting the websocket but unless we log in so let's go ahead and try to fix the backend now so that we actually accept the logins from that location and the first thing I think we should do is that we should create a new file and that file will be containing our one-time password solution I'm going to go ahead go here create a file called odp.co at odp.go will contain everything related to the one-time password so let's go ahead let's create a ODP struct and we will have a key and we will have the time the one-time password was created and let's also create something called a retention map that's called retention map and this will be a map with a string as key and a OPP value so the idea here is that this map will contain the one-time password key in the map and the one-time password as a value and the retention map will delete any one-time passwords that are too old so let's go ahead and create a Factory for the one retention map we will accept a context and we will accept something called a retention period which is the duration for how long a one-time password is valid after the time has passed we should remove the one-time password and no longer make them uh legit so let's create and return the retention map we will soon add the actual retention so let's go ahead retention map we can do new OTP and new OTP will return a one-time password so let's go ahead and this solution will be using Google uu ID it's a package which makes it fairly simple to create unique strings in our real life solution I will probably strengthen this a lot I'm just making a very quick solution right now to show you how the ticketing system will work so we we create a function that allows the retention map to create new one-time passwords and the created will be time.net and we will add this to the retention map so let's go ahead and do a retention map one let's use the key and let's place it as value and let's also return it now we also need a function which will verify that a one-time password is actually valid so let's call it verify OTP it will accept a string and return a boo so if this key exists we will return true so let's check the retention map for the one-time password and if it doesn't exist we will return false so OTP is not valid and we will remove the one-time password and return true if somebody uses it so verify one-time password will accept one a key check if it exists if it doesn't exist it will return false if it does exist it will delete it it's a one time password so we should delete it if it has been used and then we will return true so we can create new passwords and we can verify them and delete them let's also go ahead and start their attention so retention is a function with a context as input the retention period uh the second parameter and we will create a ticker so time dot new ticker and the ticker will be let's say each 400 milliseconds this is how often we're going to be rechecking all the one-time passwords that are out there so this will be running as a go routine and each time the ticker ticks we're going to Loop through all the one-time passwords and we're going to be using the timestamp stamp when it was created we're going to add the retention period to know the time when the one-time password is no longer valid now we're going to see if that time is the 4 time not now so the expiracy date is before our current time this means that the password is no longer valid so let's remove it let's also listen to the context done and return to cancel this function whenever the context gets close so let's go back up here whenever somebody creates a retention map let's just go ahead and create the background process that runs so go remove retention will start a go routine which runs in the background which continuously checks for expired one-time passwords so we have a solution to create one-time passwords and delete them whenever they're old now that's great now the next part we need to do is we need to jump to the manager and the manager should have one of these retention Maps so let's go ahead and add that let's call it otps and it's going to be a retention map and in the factory new manager function let's go ahead and simply create a new retention map we're going to add the context and we're going to say five seconds is the retention period now we don't have a context in the manager so let's go ahead and add that because we will need that for now so I'm adding a context to the manager also and the next thing we need to do is we need to make a new login endpoint um we have no slash login endpoint right now so we're going to create a super simple Handler which the manager will have and let's do it here under the serve websockets so it's going to be a function for the manager which is let's call it login Handler and it's a HTTP Handler so we need to have a response writer and we need to have a pointer to a request and we're going to accept the username and password remember as Json so let's create an inline type called user login request extract it will contain the username and it will be username in Json it will contain the password so that's how the request will look so let's go ahead and initialize a new object and let's decode the Json data that is coming inside the request body and we want to decode it sorry that's supposed to be new decoder let's create a new Json decoder that's passing the request body let's recode it into decoded into our requests so let's go ahead and check if error is not nil then we're going to return HTTP error uh something went wrong maybe they have a misformed request uh request but let's return that to the user let's be nice all right we we're accepting the Json data we're decoding it and we're returning an error if anything's wrong now we need to somehow tell the user that they are logged in or we need to actually log in the user the solution I'm going to be implementing is quick because we're learning how to do this not how to implement an authorization to do that you can check another guide that is for too big of a scope to do in this tutorial I'm going to do a hard-coded login whenever the username is firstly whenever the password is one two three we will accept them you should replace this with your real authentication mechanism so let's hard code that and let's say whenever we're logged in we're going to respond and the response is going to contain the one-time password and it's going to contain it in the Json field OTP because that's what we said inside the here in the JavaScript we are expecting OTP let's go ahead and return that to the user we also need to of course from the manager going to the retention map and then do new one-time password and let's create a new response and let's say that's a one-time password is the one-time password dot key so we're only returning the key let's Marshall this and let's see if there's any errors when we Marshall let's speed this up a bit so I'm just gonna log whenever we have a Json Marshall so let's write the header we're going to write a stock that's okay to the user and we're going to write the data and return and if we get here if it's if the password and username doesn't match our amazing secure password we're going to return status unauthorized so this is our login Handler it's fairly simple we accept the request we decode it we check if it's these hard-coded values and then if the user is logged in we create a new one-time password or a ticket so-called if you wish and we return that to the user now I hope that's fairly understandable and one last thing we need to do is we need to go into Main we need to create a um so one more thing we need to do since we're sending the one-time password to the user and they are sending it back whenever we are serving the websockets we need to actually check if the one-time password is valid and we want to do that before we connect them so at the top of the serve websocket let's go ahead and grab it the ticket from the OTP get parameter so if OTP is empty let's go ahead and tell them that they are again unauthorized then we have a verify method remember so let's go ahead and if otp's verify OTP let's pass in the one-time password and if this returns false let's just simply go ahead return unauthorized and that's it if they get here they have a valid ticket and we will allow them to connect so one last thing that we need to do is we need to go into the main function we made the manager accept a context so we need to actually create a context and let's pass that context in to the manager we also need to set up the login endpoints so HTTP handle func slash login and it will execute the login Handler once that's done we are all set we now have a authentication system in place now I know we are using hard-coded passwords which is bad you should replace that with your own authentication logic so let's go ahead let's start up we can see that the websocket is connected false let's go ahead and pay Percy bomber one two three and let's log in we can see here that we received a one-time password received the one-time password and a 200 from the login however something is wrong here we didn't add the websock the one-time password which means we weren't allowed to connect so let's go ahead and check the JavaScript where we go wrong let's go into the JavaScript the login function to see where we're wrong and we're passing the data inside ah right I'm using a comma here it should be plus so go ahead and change the comma to plus because we want to append the OTP to the string let's visit the website again and go to the network tab let's clear it let's login and let's see what happens so we are now connected to the websockets perfect I hope that makes sense it's pretty easy it's a pretty easy scheme you know you authenticate over regular HTTP you get back at one time password which is used as a some sort of authentication ticket or token or whatever solution you can do is that you can have each message or each event sent with a JWT token or whatever but I try to avoid that because it becomes pretty messy that each event should contain that and you could probably make this wrapper event which has it but again you need to send the token over and over again this prevents us from having to do that and it also prevents Bots from connecting and just consuming resources so we've added a bunch of security stuff we have authentication cross domain we have a jumbo frame fixed there's one thing that's left we're using unencrypted data uh in web sockets you can Define which protocol to use and we're using the Ws protocol which stands for websockets it's the equivalent of HTTP now in production you will want to use WSS which stands for websocket secure and it's the same thing but websockets over https that means our traffic will be encrypted it's very simple to set up so you should really do it so let's go ahead let's close the program the third first thing we need to do is we need to change the protocol which is used which we Define in the beginning of the URL it should be WSS instead so that's websockets secure that's all you need to change in the front end now in the back end we need to host the server as https and this will require us to have certificates and I'm going to show you really quickly how you can fix it if you don't have a certificate you can create a self-signed certificate using openssl and you can find open SSL installation instructions on their GitHub there's going to be a link in the description I won't show you how to install it it's fairly easy it's like takes a minute or two now we're going to create certificates using openssl so to do that I like to I have this bash script actually that I always keep handy so I'm going to do some bashing um you can do this right in the terminal I have to do it here to so I don't have to redo it I'm not going to explain everything this is really advanced stuff just bear with me um so we're going to create create a server key which we can use and we're going to use openssl gen RSO to create a RSA and we're going to Output it as a server key we're going to use 2048 bits let's do X param let's do gen key and we're going to name we're going to use the effect key algorithm 38 I'm going to check out my sheet sheet here so I'm going to Output it at the server key so we've generated yeah now we also need a server certificate so server.sert let's do open SSL new so there's multiple ways you can create self-signed certificates you and I'm using Linux so this is the easiest way for me you can probably look up how to create certificates using using open SSL for your operating system if there's any differences so the batch script is only these three lines actually and whenever you execute this it will create two files it will create the server key and we have a typo let me see open episode wreck it's like six key owls all right they should have and I missed that sign so once that's done you should have two files server search and server key um now these files we can use to encrypt traffic also remember to add those files if you're using certificates in your code fetch the certificates from someplace safe don't push them to GitHub there are Bots out there that will scrape your repository really fast and steal your keys get ignore them don't store them in your repository whatever just keep them safe right once that's said and done let's go to the main function and we're using HTTP listen and serve we're going to change this we're going to change this to https and that's done by using listen and serve of PLS instead so listen and serve is regular HTTP listen and serve TLS accepts the same parameters but also a certificate file and a key file and you've guessed right it's the two files that we created first go ahead and add that we're going to say server search and we're also adding the server key and then nil so that's all you need to change now actually that's not everything let's go into the manager and let's search for reading checker our origin will no longer be HTTP localhost it will be https because we're now using https right so that's all you need to change to make this work using https right so again we can go ahead and we can go around this it will set up the https API instead we can go to the front end again which we have changed to use websocket secure instead all right so also I'm going to httpf localhost your connection is not private you will see this error message in your browser and this is because you're using a self-signed certificate if you're using a real certificate which the browser recognizes you should be fine however I know that I'm using a unsafe certificate right now so we can click Advanced and proceed to localhost and here I am back inside this let's go ahead and try to connect again and we will see that we're using the websocket secure instead and everything is working as expected let's type something let's send a message let's see that it's sent so great we have a working Solution that's um doing um everything that we did before but we're now having a secure encrypted traffic which is really important um now you might also see this error here handshake error and this is a remote error this means that it's printed from the client this is telling you that the browser does not recognize the certificates but it's not an error in that way if you use the real certificate you wouldn't get that so you have to ignore those messages for now and congratulations you're using encrypted websocket traffic instead of online encrypted so we have covered how to connect websockets how to send messages properly how to heartbeat how to apply a bunch of security now there really isn't much left with what we've learned today you should be able to build a production relative full blown API but before we conclude this tutorial I want to make us implement the actual event handlers so that we can make the chat properly work I won't be covering anything more architecturally specific about web sockets I think what we have here is very sufficient to have a production radio system we will only get some hands-on experience by finalizing the handlers it won't be too much so let's go ahead and Dive Right In let's update the events that we can send so let's begin inside the manager and let's go to setup event handlers where we currently only accept the send message and send message simply prints the message to the console let's fix that there's a nicer ways to structure the code I'm just going to go ahead and do this simple for us so whenever we receive a new message we want a structure for apps we can actually place the new message event inside of the events so new message events and it will contain the send message events okay or inherit that but this will also contain the time the message was sent so it will contain message from and also the time the message was sent inside send message let's go ahead and create an instance of the that so we have a shadow event which is send message event and we will Marshall the event payload into the chat event because that's what we're receiving right okay share event and we will check if the error is not nil and if it is we will return a wrapped error saying bad payload in the request and we'll do that now whenever we send a message we want this message to be sent to all other clients so let's go ahead and create a broadcast message which is in the format of new message and Broad message will have the send timestamp to the current time and the message field will be the message from the chat event and the from will be the same from as the chat event from so let's go ahead and Marshall this data we will after we have Marshall lit we will send it out to the other flag so let's check if the error isn't new and let's return a wrapped error saying fail to Marshall the broadcast so we have our broadcast message as a payload and let's create the outgoing event and we can do this outgoing events equals event and the payload will be our data and the type will be our event new message type oh new message oh and then new message it seems we don't have that yet so it's going inside events and then new message so what's going to be like new underscore message so we create a new event type and we say that the outgoing event is of this type to differ between the send event and the new event uh send messages and in new messages the send messages is when the user sends the message new message is when they receive you could probably use send and receive to make it up a little bit easy here so this broadcast message or outgoing events should be used and we will simply iterate all the clients connected to the manager and we will do the client egress and send the outgoing event so the send message Handler is really simple we receive a send message event from the user they type in their UI send this message click the submit button it gets here we Marshall it we append data to it we append the timestamp that they sent the message we Marshall it and then we send this new data out to all other clients so that's all we have to do on the back end to implement the front end we will change a few things and we already have the event class here I'm going to add two new classes so they're going to be the same as inside the backend so send message event and we're going to add a Constructor and we're going to accept the message and from because that's the field that we expect in the back end so let's just assign them whenever they we create a new send message event and then let's go ahead and do the same thing for new message events and there are also going to be the same as in the backend so let's again go ahead and do the signing so we have two classes that corresponds between the back end and the front now the send message function here we're going to update it to send the proper format and to do that we're going to go in here and say let's outgoing events New send message events we're going to pass in the data so new message dot value we're going to send from who it is and let's hard code this to firstly because we have no form field where we allow them to set their username so let's go ahead and just do that we create a new send message event with the value of the message and a hard-coded value of the username and then we send that event to the websock um so what will happen now is we will send the send message event that will be accepted by clients and pushed out to all the other clients and even yourself because we broadcast it to all clients um it will return a new message event which we are supposed to render so let's go up to Route event and you can see I prepared the new message here but we're not going to print new message simply we're going to do a message event and we're going to object assign new message new message event and we're going to assign the data payload and let's see after we have the message event so this will take the payload create a new message event from it and we want to add that to the chats message so let's go let's create a new function which is called a pen shot message which accepts a parameter which is the event so let's create a new date from the message event.send so the payload that comes in will have the sent field with the date that the message was sent so let's create a date from that and let's format the message let's format this very easily let's do dates to locate page string now let's do local string and let's have the message after so we will print them date and then the message and let's append this to the text area so let's fetch the text area document okay element by ID and it's called chat I think shot messages that's correct so let's get this chat messages and let's go ahead and do a simple inner html text area is inner HTML plus let's add a new line for each message and then the format the message let's also scroll the text area because we want it to follow along the messages that are sent so scroll top equals text area scroll height this will kind of keep it scrolling so we are sending messages we're broadcasting those messages and we're printing the messages to the text area let's go ahead and try this so let's start up the back end and let's open up the front end and let's go ahead and open two tabs so let's log both of them in we're connected and this one is not connected but let's connect it so we have two tabs which are connected let's go ahead and do place them next to each other like this so we have two connected clients and let's go ahead and type hello in one of them and send it and we actually have an error error handling message bad payload in requests cannot on Marshall string into Go Value so we cannot unmodel the string into a go value and if we go into the send message we can see I forgot to update the send message event we're sending the string value we're not sending the outgoing events so let's update that and let's connect both clients again so let's go ahead let's kind of refresh both I'm going to connect both of them and they are both connected so let's send hello and we didn't see the message all right so we need to also call a pen shot message and let's append the chat message events so once we've done that we should now see each new message being sent let's go ahead update that let's open both our tabs and let's refresh them and let's go ahead and log in let's go ahead and log in both of them are logged in let's send hello and you will see our own message but the other client will also see the Hello message so perfect we can now send messages between each and every client that was easy so we have this amazing shaft where we can send messages to each other and if we want even more practice we could implement the whole chat room stuff right now we're using one chat room but you could easily update this by adding a new yes um new event and a new event handler let's take a minute and do that just to show you how easy it is to add new features so we create a new event called Shane chat room and we're going to create the Constructor which will receive the name of the chat room and let's go ahead right here do function change chat room and we're going to get the chat room currently selected get chat room and if new shirt isn't null or if the value isn't equal to the currently selected chat because we don't want them to spam change to the same chat room you can only change to a new chapter so let's assign the currently selected chat room to the new chat room and let's create a header um documents get element by ID so we call it the chat header and we're going to change to currently in and then just like let's apply chat room let's just apply the selected chat just to show the users which chat room they are currently in now let's send a message to the websocket that we are changing the chat room and let's create the event let's create the event by inserting the selected chats and let's just send the event it's called The Chain div and change room and we're going to send the change events which we create there send it and let's also clean the text area let's go ahead and copy the text area text area selection and let's just make it empty inner or Let's do let's do a format that you changed room into just to show the users what they're doing you changed into the selected chat room so that's everything from the front and oh we also whenever they go uh whenever they submit in the here we can see chat room selection we use this change chat room function so in here we also need to change the chat room so let's grab the change Shadow function oh we're doing it this is my bad let's simply we have two functions so my bad let's remove this and only have the this function the new function been update if I created a new function instead of updating the old one sorry so we have a change chat room in place which will trigger a event so let's go inside the back end manager and let's go to setup event handlers where we can add new events and we're going to add a event called event change room and let's do chat room hander as the function let's go inside the events we're going to create a new event called chat room and we're going to use the same string value as we set in the JavaScript which was change room let's save and let's update the clients as well the client will have a field called chat room which will be used by the client to kind of know which chat room they currently in on the back side and back-end side of things also um so inside the event again we have added the change room and let's go ahead and add a Handler for this so let's go ahead a add a change room event and the event we created in the front and simply passing the string name so let's just accept that and let's go into the manager and let's go ahead and do so let's go inside the manager and let's create the actual chat room Handler function and it has to be a Handler signature so we're going to accept the event and the client who sent the message let's go ahead and save and let's see events change room does not exist what did I name it Shane true perfect so the Handler will be really simple let's accept the exchange room events and let's on Marshall the payload which will contain the name of the new chat room that we're going to be inside and let's see if there's any errors but payload in requests so once we received the change chat room event we will take the client because we accept the client here we will simply take that line and change the chat room that is assigned to the event name and we will return new so this is a super simple Handler which will change the chat room that the client is currently in now we also need to go down to send message and where when we Loop over the clients we can simply add a if client chat room equals um the current client Etc so somebody sends a message we will only send that message to the same chat room that the current client is in so we can see that if each client who is in the same chat room as the current client we want to send a message to their egress now whenever we're in different chat rooms we won't get the same message perfect let's restart the back end this should be everything I think so let's open up our two separate front ends again let's reconnect them and let's log in once they are both logged in we are logged in we're both in general so let's do one two three both of them receive it let's go ahead and change to the private chat room and something failed let's go ahead and change to the let's login and let's see what's going on change chat room and make this bigger so it's actually sending a regular HTTP request so something is wrong oh sorry we forgot to return false always return false if we don't return false it will redirect since it's a form apart from returning false we also need to change that to assign name into name not name into this let's go ahead and restart the application bring up the front ends again and let me make this smaller and let's update both of them and let's log in that is logged in that one is logged in let's send a message and we can see both of them receive the message but let's change this user into a private chat room so we've changed into a private chat room and let's send hello again and we're the only ones to receive it now this one is in another chat room so it doesn't receive perfect now this concludes the tutorial we have built a whole framework for how to use web sockets in a server with go back and we have a server that's secure scalable and easily to manage according to me so we have covered the following aspects of websockets in this tutorial we have learned how to connect the websockets how to efficiently read and write messages to websockets in go how to structure I go backend API with websockets using the events based design pattern you can clean up the current structure in the current code a lot it's all in the same package for instance we could separate that that's basically up to you we have learned how to connect key connections alive in websockets by using ping pong and hard beading we have learned how to avoid users from exploiting the websocket by limiting message size and also how to only limit the origins they come from how to authenticate using an OTP system you should be able to replace that authentication with whatever solution you want uh how to also encrypt the traffic and use https and wsf instead of regular websockets so I hope this tutorial has helped you understand only at least a little bit more about websockets in go and you should be you should easily be able to Port this and create clients for whatever language you want um if you want to use another go client instead of a JavaScript client for instance you can use the same ideas and methods that we have learned so if you have any ideas feedback or stuff you want to know more about please feel free to reach out to me uh either in the comments here at YouTube or in the contact me page on my website um and I really hope you enjoyed this article I know I enjoyed creating it so give a thumbs up comment and subscribe to my channel if you want to see more of me

Channel: ProgrammingPercy

Views: 19,557

Keywords: websockets in go, golang websockets tutorial, golang websocket, javascript websocket, javascript websocket tutorial, go basic websocket tutorial, go websocket gorilla, go websocket tutorial, go realtime websocket tutorial, go gorilla/websocket tutorial, websocket, coding tutorials, javascript, javascript tutorial, programming tutorial, go programming tutorial, go programming language, golang tutorial, authentication websocket, websocket secure, wss, websocket https

Id: pKpKv9MKN-E

Length: 144min 34sec (8674 seconds)

Published: Tue Nov 22 2022