Scaling Websockets with Redis, HAProxy and Node JS - High-availability Group Chat Application

what is going on guys my name is Hussein and in this video I want to talk about how you scale WebSocket connections guys to run on multiple servers and I'm gonna show you how first illustrate that how WebSocket works I am going to illustrate how you scale WebSocket connection with a reverse proxy such as a a proxy or nginx and once we do that I'm gonna show you exactly a demo or some some code that I wrote that simulates a live chat application that is scalable with docker right so I'm gonna spin up a little bit for multiple servers with the multiple WebSocket servers and one Redis connection and one reverse proxy and we're gonna distribute all this beautiful connection WebSocket II stuff and that's gonna be fun right how about we do that right you can skip to the code part if you're interesting I'm gonna put the jump code below that's it let's jump into it alright good so a whoops whoops sockets connection here is how WebSocket connection work I made a video about WebSockets go ahead and check them out if you want details but in a nutshell if you have a client here and you have a web server just a normal observer the first thing that the client does is it tries to establish just a normal HTTP connection and it just hey I want to do a get request but that get request has a something very special in it it has something called the upgrade header right and that upgrades here is like hey whoop server I know we just established a TCP connection for the HTTP session but is there a chance I can upgrade that to a WebSocket the server says wait a second upgrade WebSocket yeah I do support that right go ahead and let's reuse this connection for WebSocket from now on that HTTP session is now hijacked just for WebSocket now this is only true for HTTP 1 1 this is not true in case HTTP to HTTPS there 1 stream of HTTP 2 1 almost like one thread of HTTP 2 right the stream will be used for WebSocket connection because as well it will be just waste to waste the whole TCP connection just for a WebSocket right but that's another topic for another day all right so what we're going to do is like the server now can send information just randomly to the client the client can send information to the server's just a bi-directional thing right and that's how WebSockets generally work right so it's a very stateful thing so we're using we're aware of this connection we are aware of this connection and the client just sends information to the sky Church a chick's the state of the connection right so how do you scale something that is stateful right with HTTP it's very simple right because it it's just make a request to the reverse proxy and the reverse proxy can load balance and hit that request to any web server at the back in and they will be able to serve us what we shot gets a little bit different right so let's take that same example with WebSocket and here's what we're gonna do I am going to simulate three clients the first client is the right client right we have red okay so what we're gonna do we will have some sort of a reverse proxy here and listening certain port let's say 80 or 443 probably 443 for security reasons and that reverse proxy will establish the connection on our behalf to the back-end server so these guys are the WebSocket servers now right WebSockets WebSocket WebSockets right so these are the WebSocket server so how do i scale here's the interesting part here if the first coin made a request and says get upgrade write that reverse proxy if it's supports WebSocket and there's a big if not all reverse proxy support WebSocket connection so you need to find some someone that does right so I know a proxy does Ingenix does I think I'm not sure about the rest right so now we have a request here and it checks the reverse proxy says oh you want to upgrade the header huh I know how to deal with this let me just a second let me let me connect to one of the servers and let's say the load load balancing algorithm is round-robin so it's gonna connect you to this and now there is one TCP connection between the reverse proxy and back and the back end server and there is a one TCP connection between you and the reverse proxy right but since this is WebSockets right protocol that reverse proxy will act I can layer for proxy so it will stream every single packet that you send from now on to always go to this server that's it that there is a dedicated TCP connection to this client so it's like an almost almost act like a 1:1 TCP connection but it's really not there are some implementation that where this is physically one TCP connection so where this guy acts like a gateway just like your router but this is outside the scope of this video but let's just say for simplicity you now you understand that right ok let's take my second beautiful yellow client my yellow client here now makes another TCP connection well once a WebSocket right so that reverse proxy would say ok round-robin I the first request was to this guy so now the second request goes to this guy and so on right so let's just go the blue guy sure this guy this is that WebSocket connection so now let's just simulate what would happen here if I start sending a request right that's what happened if I start sending so let's say the server start sending information to the client right so we'll start sending information the reverse box you say oh whereas again this is part of this so it will link it somehow to this right so it's look almost like one connection any request that you sin will go to this point now this coin if it sends information to that TCP connection it will go through that pipe right and let's say the client the yellow client now sends information here it will go through this and the blue will go through well this right so frequent request will not be load balance that that's that's what you need to understand what WebSockets because you cannot just they and suddenly jumped to go to the other server because there is state here guys that a state that these guys agree on like if you're building a chatting application or a game there is a state that is here you cannot just abandoned that state and just jump to another one you can but it's it's really really hard to build right it's just it's almost impossible to do that right so you do that and that's the idea that's how you scale WebSocket connection so you're this guy's acting essentially as a layer four load balancer but that configuration is technically a layer seven because why because this guy actually looked at the header and says oh what uh this is up a great connection how does it notes upgrade it looks at looks at the header in and sees right so and I'm know I'm I'm I'm omitting a lot of details guys I know this is probably this guy is a TLS Terminator where it actually has a secure certificate serving the certificates but at the backend you can either be secured or unsecured really up to you guys all right all right guys to the fun part the coding part so I'm going to say spends a few minutes explaining the architecture of the application the live chatting application that I built and then we will essentially just show you the code how about we do that guys all right so here's what I have here I have a Redis instance here right and I have the same thing these guys W whoop circuit servers right it's a bunch of WebSockets servers this is the reverse proxy which in my case is H a proxy and these are the clients so these clients want to communicate with each other right they all entered at like a lobby and they want to start chatting with each other so that's the application so the red guy connects obviously which will obviously be streamed to one of those servers and that's it now one TCP connection almost like a logical TCP connection between the client and this server one okay let's take the second client which is blue in this case communicate it's the second server and the final client is the Green client so this guy connected this guy and this guy's to connect to this guy right and that's that's absolutely fine so the first problem here if this guy started to send chat messages says hi right how do this hi propagates to this guy and this guy the client number this these two clients right these are completely different servers right so here's what I did right I used the rightest as a pop subsystem in this case I know I talked about the Reuters guys check out the video I made about writers and WebSockets III really generally encourage you to to read about this back in technology it's really pretty pretty cool technologies right I'm gonna use another color here for the back end and here's what I gonna do I use the pops up technology I know we talked about pops up in this channel guys right and what we're gonna do here I created a channel called live chat right and I made all these guys when they spin up I made them subscribe to this live chat channel to this channel okay and what does it mean to subscribe that means if someone published something to this those three guys will see it immediately and then here's what we're gonna do so this client sends a message says hi which will be transferred to this guy which then is responsible to publish the hi message to their live chat which will the Redis code will trigger and it will say okay who's subscribed to live chat oh this guy this guy's this guy so this guy will get it this guy will get it also this guy will get it as well and what will happen is like okay there will be code that would say whenever we receive a message from the channel go ahead and push it to all my clients so in this case hi-hi will be transferred back to everybody so well this is the coin this client will get the message this Gordon message also this client will get the message because it will just give his message back right because we don't know right we just subscribe and the message will be distributed to all of these guys how about we actually show you the car code that's most part it's really really simple because I just use docker for the whole thing that's it you just need docker and visual studio code to write the code how about we jump into it so here's the code goes I'm gonna go through the docker compose and guys you might say what the heck is docker compose check out the video I made here micro-services and how to spin up all this stuff so the first thing is the load balancer which is this puppy right and I'm gonna use a che proxy and I talked about is your proxy in this channel go check out the course I built and share proxy and we're gonna let an import 8080 and here's the a proxy configuration we'll go go through that and what we're building here we're spinning for or free actually for haha for WebSocket servers right and the WebSocket logic server is actually in this image which we are about to show that that's the code we written that Oh connect to this Redis understand sign and publish and connect and send information back so all this logic lives in there and the WebSocket the server so and I am just sending some app ID here so we know which server were connecting to it's just mumbo-jumbo nothing fancy here and then finally there it is server that does all this magic right and here's the host names these are host names essentially okay and now go let's go to the HF proxy configuration the HF proxy configuration you see large timeouts and these are very very critical because the HR proxy will terminate the connection if nobody is communicating for X amount of time right and we don't want that because this is a stateful connection and there will always be some people not sending messages for a long time right so we need to keep this connection open so what do we need to do is like configure the backend and this back-end will have four servers server WS + WS 1 w s to W is 3 the video is for listening on port 80 80 80 80 right and this is just like logical server names that is for the configuration alright and what is the final piece the docker file which will build our application all right this will be building a node.js application that will have the WebSocket server which we discussed in another video right how to build it this is then nodejs application itself we are using HTTP module we are using the WebSocket module obviously we need Redis as well and we're taking the app ID and here's what we're gonna do we'll create two connections from the WebSockets a subscriber connection and a publisher connection and that's too bad because I don't like this but I think I guess this is just the limitation that I this team have right because this is what we're talking about because this guy is also a publisher and a subscriber you need to TCP connections and the reason is the moment you subscribe you cannot publish on the same TCP connection for limitation reasons obviously quick will definitely solve this problem I have no idea if this team will ever move too quick but that's definitely one solution right there will be one TCP connection well UDP in this case but a stream will be reserved for a subscriber and the others they will be reserved for a publisher I think that's a great solution rabbitmq came up with the idea of channel so I don't know why the writers team can also implement that channels in the TCP right so they need to augment the rest protocol to do that anyway sorry about guys because I just ramble a lot when it comes to these technical details I love this stuff guys all right so we'd have publishing software for each one unsubscribe if someone subscribe to you just immediately publish a message that's just garbage there's nothing really special about this but if I receive a message from the subscriber this is this way right if I'm going this way right from the writers to this if I received a message go ahead and loop through every single WebSocket connection that I have and send that message to my subscriber so in this case I'm gonna send it to the reverse proxy the reverse proxy obviously is smart enough to serve it to the right client right because it's although this logic is lives in probably the net right Network allocation table I might be wrong there and we're subscribing to the large Chuck and on the radius server okay we created this is just a WebSocket logic connection we created a bland TCP connection and then inherit that logic in the HTTP server listening on port 8080 that explains why we have all bunch of 80 80s here right and on request this is the connection upgrade if I am open close or do this events if someone just sent me a message do this right and publish that message to that Redis server on the live chat channel channel is very overloaded huh channel on RabbitMQ means differently completely different thing than a channel on writers all right so said timeout obviously and this is just a timeout that whenever your connection after five second I will tell you hey you're connected to server one or two or three and then you can as obviously have many connections so I'm going to add this connection to a pool of connection array that I have here and that's it and that's the client code how about we actually just thank guys so to do that we need to do two things let's clear all my previous attempts here we didn't gonna do docker - docker build - t WS app is that what I called it WS app that's the image right so WS app and then dot this will build the late that's the first thing you need to do build the docker image for the web socket and we'll build it from basically from this guy right the second thing we need to do is docker compose up that's it that will start spinning up all those beautiful WebSocket servers all those beautiful heo proxies and Redis and everything so it looks like we're up and running how about we start testing this thing I'm gonna open a brand-new chrome I'm gonna go to the developer tools clear that thing and here's what I'm gonna do Const this is zoom in here so you guys can see that value is I'll just say WS because I'm gonna do this a lot obviously equal new whoops socket WS localhost because it's I'm running on my machine right and it's exposed all right I said and then I just connected to the server we need to set up the message obviously and then finally we need to start sending so this is client number one I'm gonna say hello I am client number one okay and immediately I got the same message obviously we have one client so we just need to do this all over again on another client all right guys let's do this again alright guys now I'm open to Chrome the views here I'm gonna establish a connection right here and then I'm gonna wire the message and I'm gonna send hey I'm client number one immediately I got it and the 5-second immediately got it home you'll connect to server for right by the way this is all around drop in but let's check now I'm gonna connect from Klein to here alright and since I am Klein number two I am connected to Clyde number one this case all right look at this we got that message now if I stew that video is that send message another message immediately I got it here and I got it here despite us connected to different servers right and you can scale this to as many as you want I can open many many servers many web pages like this and I start playing with this thing I know I want you to play with this thing now so now if I send a message from here if I do WS don't send hey immediately I get here and I get it here despite me and the other client are a completely different server because we just proved it right we were connected to server number one here container number one to be specific and this guy is connected to continue number four four four and we communicate together with the power of all riders right in this case alright guys and final thing before we into this videos guys that yeah obviously we have read this as a kind of a central point of failure here but right is support clustering and replication to allow us to scale right so what are you gonna do is I could spin up multiple writers but this is in order to essentially scale that but I rarely really really doubt that you're gonna need that except you can have a thousands and thousands of connections scaling just just this layer is enough to me to scale over socket connections alright guys so that's it for me today I hope you enjoyed this video WebSockets have a great weekend I'm gonna see you in the next one you guys stay awesome throwing all your questions that you have below what should I make next I am going to see in the next one you guys stay awesome

Channel: Hussein Nasser

Views: 23,158

Rating: 4.9433718 out of 5

Keywords: software engineering, backend engineering, websockets scaling, scaling horizintally websockets, redis websocket, scaling websockets, redis pub sub, redis publishing, redis scaling, redis websockets, redis pub sub with websockets

Id: gzIcGhJC8hA

Length: 20min 35sec (1235 seconds)

Published: Fri May 15 2020