Load Balancing with NGINX

[Music] hi guys in this video we will learn how to load balance our new GS applications with a half of nginx so right now our DevOps landscape looks like on the picture here nginx that is in the middle is responsible for doing several important things firstly it terminates HTTP traffic so it handles the encryption and then forwards all the traffic through the unencrypted channel to no GS application this way no GF doesn't have to deal with certificate and a decryption itself secondly nginx is responsible for serving static files so what we want to do in this video is to give nginx one more function so now it will not just send traffic to a single no G as instance but instead it will be sending traffic to multiple no GS instances running exactly the same application this is called a load balancing so each no GS app will have its portion of load and its portion of users to serve Before we jump in and start fiddling with context let's figure out why do we need this load balancing thing at all the first reason to implement load balancing is well to distribute the load to balance the load as you know new GS is single threaded in its core so if you have a multi-core or multi CPU server the only way to fully utilize the potential of that server is to have multiple instances of new GS running but in our digital ocean server we right now have just a single CPU core this is one of the cheapest instances so there is not many cores to play with and you might be asking why do you need load balancing in this case well load balancing has a second very important feature which is failover so what happens if nginx detects that one of the new GS instances goes down it will immediately redirect the traffic to the other instance so while PM 2 is busy restarting responding the instance that went down your users will still be able to use the other instance with this deployment strategy your application will be significantly more fail-proof last but not the least this kind of architecture will open the possibilities for zero downtime array deploys so whenever we redeploying the application we can bring the application instances down one by one so there will be at least a single app instance serving two users at any given point and I will show you how to implement this kind of zero downtime redeploy strategy in the upcoming video for the sake of this video I have updated my little easy i/o application I have added the name of the node here so H no GS process will have its own name that I can pass as an argument in the command line and also as an argument I can pass the port number to bind to so this way it will be easy for me to start multiple easy i/o processes on the same host and there will be no port conflicts just if you're curious I used minimus in order to parse command-line arguments and as usual all the code for this app is I'm github so if you decide to give it a try please go on download the code and check it out so I'm back again on my digital ocean host the first thing that I want to do here is to update p.m. to running configuration so as you see right now easy i/o is just a single instance that is running right now but instead of having just one easy i/o instance listening to a single port I now want to have multiple instances running so p.m. to stop let's just say all and p.m. to delete all to remove all those applications from p.m. to management so now p.m. 2 is not managing anything now instead of having one instance I want to launch two instances and I'll do this p.m. to start and I'll give it name easy - one and it will be mangy as still the same file however I now want to pass several command line arguments to this running applications and to do that in PMT output - - here and I can posture the argument surname will be um II V one again this argument goes to my program to my main GF this argument name is e one goes to p.m. - this is what it will display in its little grid so don't confuse this one so and the port we'll be 80 80 it is 80 80 by default I programmed it this way but I just want to be explicit here and put it on port 8080 let's launch it it looks like it is going online now since my application is listening to exactly the same port as before let's just make sure that nginx is still serving our files properly an application is still available so you see here now it's telling that the name of the note is easy one so you see socket IO is reporting the name of the note and it will be important once we start working with load balancing and if you click get name it will be easy - one forever okay now following the same pattern let's launch another instance on port 8081 and instance will be called easy - and here it will be easy to again so hit enter now we have two applications running but since nginx doesn't know about the existence of second instance it will get no traffic so it's a great time now to update nginx configs so I'll become a root again logging off from my user and now let's find our nginx configuration for nanogram IO and let's come here ok so in order to tell nginx that we have multiple nodes we need to add one more block to this config on the level of servers so not inside of a server but on the topmost level here we add a config block that is called upstream and we need to give this upstream a name so in our case it will be easy IO and this name can be anything doesn't really have to be easy IO so any name will do so inside of an app stream you have to describe the servers that can handle the traffic for this application and for me there will be two servers so server number one is localhost 8080 server number two will be localhost 8080 one so the next thing to do is to tell nginx to use this upstream and when it needs to redirect the traffic so where are we redirecting traffic here proxy pass here are two directives where we are rejecting something somewhere so let's go here and now instead of using localhost:8080 we put the name of upstream here so I'll put a vio so nginx is small enough to figure out this is the name of upstream it is not a name of actual host so it will take one of the server's described in this easy i/o block and direct traffic's there and we'll do the same for the API and let's remove localhost and add easy IO all so let's make formatting a little bit prettier so awesome we can save this configuration and now let's do system CDL reload nginx so in previous videos I was recommending to call a restart nginx actually this is a little bit better way to reload your nginx configuration because it will not make nginx process to stop and start again it will just tell nginx to reload the config and use the new configuration for the upcoming request so this is much more graceful way to do that and I suggest you guys to use nginx reload sorry systemctl reload instead of restart so see just one second and it's reloaded let's go back to our application and let's restart this page so heartbeats are broken but now you see already this pattern you always have to handle WebSockets separately right so heartbeats are not going anymore but I will explain why and we'll fix it in just a second but now let's play a little bit well as the name I'll get name it's easy one I'll get name it's easy to is it too easy one so as I'm clicking this button I'm getting the traffic from the different new GS instances that are running on this horse so load balancing is working perfectly except for WebSockets that are not handled at this point at all so let's try to fix this problem so we're now back in our configuration and let's look what is the problem with WebSockets again the problem is that WebSockets handshake process through the process of establishing a connection between a browser and the server is a multi-step process and obviously each of those steps each HTTP call should go to the same server however right now what nginx does it just sends all the traffic's randomly so most of the times we will be unlucky and the first part of the handshake process will go to this node instance and the second part will go to this node instance obviously handshake will not complete with this kind of situation and that's why we cannot establish WebSocket connections okay so how to fix it the easiest way is to tell nginx to send the request from the same horse from the same client to exactly the same new GS instance and to do that we just need to add one directive here that is called IP hash so depending on the IP address of the client he will be sent to one of the nodes here and nginx will make sure that all the subsequent requests will end up on the same node so WebSockets should work now right let's save it and reload nginx config now let's see that fixed the issue right so and I already see it did fix the issue so easy IO - now No - was lucky to get the traffic and if I click get name you will see that I'm clicking this button over and over again but easy IO - is the node that handling the instance if I would load this site now from the different IP address from the different host I might get to easy IO 1 or easy io 2 depending on the mood of nginx at this particular point in time right but the thing is that same class will go to the same instance of nogf this strategy is also often called sticky sessions and you need sticky sessions if you want to have WebSockets so let's see if this kind of strategy gave us any benefits in terms of failover and when you're dealing with a failover it's just best to try it with your own hands try to put no down and see what happens and that's exactly what we'll do right now so we are served by easy to note what happens if it goes down I became again my unprivileged user here and p.m. - LS will give me easy - is up and let's bring it down p.m. to stop easy - - ok so now it stops it is down it's not accepting any traffic what happens here well I was talking about it while I was bringing node down socket I already reconnected - using you know to the other node so Sokka dial traffic's remained intact and if I start getting names here I will get easy one because the second node is down engine ax is small enough to not direct any traffic there at all and your users are having seamless experience right now let's try to bring this node back up and I'll right here start easy to and we'll see what will happen so so cat eye Oh traffic will not change because WebSocket connection is established and there is no reason to break the connection that is already established so this one goes to easy one node however if I start clicking get name here you will see that I'm getting easy to the node that is back up and running so you see this kind of funny properties you can have the connection that ends up on one node and your API is our search from the second notes so if you're designing the application that will work in a cluster there will be more than one node serving the request from the user you must make sure that your application is aware about it right so it's designed to work this way for example if I do broadcast on this node with socket I'll if I don't do any additional steps it will only broadcast to the clients connected to that node but not the other node right also you cannot assume that your web socket traffic will be served by the same node as your API calls so with IP hash and bring in nodes up and down you might have a client that will end up with a picture like this so WebSocket traffic served from easy one this the API calls are served from easy to just because of the way the connections have been established last but not the least we need to update our script that does a redeploy this little shell script that sends the archive to our server and redeploy our app the one that we build few videos before so right now we are stopping and starting the application by its name with p.m. - let's just replace it - also instead of just stopping a single instance with a VM to stop all p.m. to start all again okay one important thing to notice is that this script will not give you zero downtime redeploys right so you are protected from the failures of the nodes but you are not protected from your own deployed why well because your stop everything and then while this housekeeping happens right while you are downloading the dependencies extracting archives etc your notes are down in the upcoming videos we will see how to improve the script but at this point this script will work and this is what I will commit to github together with this version of the application for you to use finally since we updated p.m. to configuration we now have two applications running instead of one we need to remember to do p.m. to save why we need to do that well because if we restart the server PM 2 will take the data from the lost save point and restore it in exactly the same way right so if I restore the server right now PM 2 will restore a single application like it was before this video if I call p.m. to save and restart my server again now it will restore exactly the proper configuration great thank you very much for watching and don't forget to click that subscribe button and like this video because in the next video we'll talk about zero downtime redeploy that will be largely based on this technique that we used in this video so keep updated subscribe to the channel and see you in upcoming videos bye [Music]

Channel: Juriy Bura

Views: 73,657

Rating: 4.9533334 out of 5

Keywords: node.js, load balancing, nginx

Id: 2X4ZO5tO7Co

Length: 14min 34sec (874 seconds)

Published: Thu Apr 13 2017

Reddit Comments

[u/chriscrowley]

I was planning on doing a setup like this. The video was very informative.

[u/beejiu]

What's the benefit of creating two PM2 processes in fork mode rather than using cluster mode? I guess Nginx is better performing, but you'd also get zero-downtime deployments directly with PM2.