Intro to Horizontal Scaling, Load Balancing, Immutable Infrastructure with AWS

in this video i'm going to go over the basics of horizontal scaling load balancing and immutable infrastructure and i'll show you how to set up this kind of infrastructure on some ec2 instances manually using aws and the concepts apply really no matter what platform you're using but i'll just specifically be showing you on aws [Music] [Music] so let's say you have a server side application running in a vm that's processing requests made by clients this app could be written in literally any programming language it's just a bunch of if statements and for loops and it takes up some processing power if we get too many requests our server might not be able to handle the load and it might just end up crashing but we can horizontally scale this app just by making a copy of the virtual machine and distributing the request between the instances using a load balancer now we can handle twice the load and if that's not enough we can just add another instance the more instances of the app we have running at once the more requests that we can process it's simple but what happens when there's an update to our code the developers add a new feature and push an update and we were on version one but now we need to update to version two how do we update all of these instances well we could go through each instance one by one pull the new code install any of the dependencies run the updates and restart the app on each instance but this can lead to some weird unknown errors and unknown states where part of the updates happen and the other parts don't and maybe something runs in a weird way or maybe the entire application crashes because the update was unsuccessful this is where immutable infrastructure comes in where we would never update an existing instance an existing virtual machine once it's there it never gets modified and when we need to make a modification we just create a brand new virtual machine image that's configured with the newest version of the application and all the dependencies and any other updates that need to come we can then create brand new virtual machines from this image and tell the load balancer to just start sending traffic to the new instances instead of the old ones then we can shut down the old instances and repeat this process every single time the code gets updated and this kind of process is really easy to implement in cloud computing environments this idea is pretty simple and this whole process can be automated by a variety of different tools and in the next part of the video i'm going to show you how to implement this kind of thing manually without using any of those tools or scripts and this can be kind of tedious and realistically you would never do it manually you would automate this process using tools like packer to create the images and terraform to actually set up the infrastructure and i might make videos on those in the future but for now i think there's a lot of value in just learning how you can do these things manually learning the process of how it's set up and then you can appreciate the automation tools a little bit more when you learn how to use them so i have a really simple web server setup on my local machine here that basically just serves up a random emoji every single time you refresh the page and what we're going to do now is deploy this to a bunch of ec2 instances horizontally scale this application and set up a load balancer to distribute the traffic for this application so we could treat this as a brand new app or we could treat it as an update to an existing app either way the process is pretty much the same so we're going to first set up a virtual machine with everything we need on it to actually run the app and we can test the app and this will just be kind of a development environment for testing that our virtual machine is set up correctly once we set that up we can create an image from that virtual machine and then create new virtual machines from that image to horizontally scale so first step is to launch a new vm just for testing the app works and then we can create the image from that so i'm just going to go in here i'm going to create a new ec2 instance t2 micro is great um when we're configuring everything i'm going to leave all of these default settings it really doesn't matter now because this is just being used to create the image that will then create the instances from so the vpc and the subnets doesn't really matter as long as i can connect to it from the public internet uh the storage default is good text so for the security group i'm actually just gonna select all tcp from anywhere because this is just a testing environment we'll use a different security group when we actually have the production servers in place that are created from this so i'll just call this uh i don't know any tcp that seems appropriate and then we'll review and launch i'll just go wait for this to be set up so the instance is set up and i am now going to connect to this using ssh so just grab the ssh details here now i'm logged into the instance and i need to just set up this application in a way that it will work in a production environment so first i'll just update the instance then i'm just going to install node and then git then i'll clone the repo and then i'm going to install the dependencies and this is a node app so i'm installing node and i'm installing the dependencies through the node package manager but this would really apply to any kind of application we're hosting we would need to pull in the code or maybe the binary we would need to install any libraries that need to run with this just to make sure that it actually runs in this environment so the server now runs so i'm just going to set this up as a service using systemd so that it will run at startup every single time and then i'll start that service so i should now be able to grab the public ip of this and make sure that my node app is running there we go there's a random emoji and it's coming from the ec2 instance and i would also want to set up anything else the instance needed so maybe i want to use cloud watch to monitor the logs i would have that set up if it's uh sending files to an s3 bucket or using rds to store data in a database that would all be set up here make sure it works correctly everything's working and then we can move on to the next step so with this app it's pretty basic everything's running so i'm going to go back into the ec2 management console and i have my ec2 instance that i've set up and i've made sure that i've tested this is working exactly as i want it to work because i'm just going to create duplicates of this instance so if this is working then i can horizontally scale it by creating you know two or three or twenty hundred instances and then just balancing load between those instances so if this is working correctly i can now go up to actions here and i'm going to go to images and templates and i can click create image and what this will do is start the process of creating an amazon machine image or an ami and this is just a vm image that i can use as a base image when i create new ec2 instances so i'll just select this image in the future and all my ec2 instances will be set up in this exact way that i have set up right now i'll just enter an image name this is my emoji image i'm going to leave the description blank i don't really need one for this there's a no reboot option that you can enable basically this will uh try and create an image from the ec2 instance without rebooting it by default it wants to stop the virtual machine and create an image that way which is a much safer and better way to do it but if we really wanted to keep our virtual machine running during this process we could check this checkbox but we probably shouldn't because it'll be much better if it's allowed to shut down the instance and create an image from the shutdown version um the default uh volume type i'm just going to leave as all of the default values i don't need to change any of those we could add volumes and do other things but this is absolutely fine and then i'm going to click the create image button and this is going to take probably like five minutes or something so take a break come back so you can take a break and then come back when it's done um i'm gonna have to wait like five minutes now but if i go over to the left panel here there should be images and under images there's amis if you click on amis you'll be able to see uh all the amis that exist in amazon including our custom amis which we can see because it's saying owned by me at the top here so this is the current image that it's creating from that virtual machine so we just have to wait for the status to change from pending to uh i don't know probably active or something take a couple minutes so my ami has now been created its status is available uh so now i'm done with this testing instance that was really only set up so that i could set up the app and test that it worked so realistically i would terminate this thing there we go i'm just going to terminate this and now we can use that image that we just created to start scaling our application and there's a lot of ways we can do this i'm going to do it the very manual way i'm just going to go create like three instances right now through the instances console but we can use an auto scaling group with an application load balancer to do it we can set it up with uh tools like cloudformation or terraform there's probably other ways but i don't know about them so let's just do it the very very manual way i'm gonna click launch new instance and now from here where we select our ami uh usually we probably select a base image ubuntu amazon linux whatever uh we're gonna go to my amis and there's my emoji image ami and i should mention uh whichever region you create the ami in that's where it will exist and it won't exist in any other region so i created this in north california which means that i'll only be able to use this image if i'm creating infrastructure within that region so keep that in mind so i'm going to select that t2 micro is good configure instance details so if i had a custom vpc right now i'd probably put this on a private subnet but i don't have a custom vpc so i'll just leave it like this i am going to create three instances though instead of one and oh that's funny i get a message saying hey maybe you want to use an auto scaling group yes i probably should but i want to do this manually uh if i had i am roles like if i was using cloudwatch or s3 or something i would set that up here like everything needs to be configured for the production environment in this case i don't need any of these to be changed so i'm just going to leave it so let's go to add storage i'm good with that storage tags i don't care security group important again because these are application instances and they're only going to exist in the private network part of our infrastructure that's where they should exist because we can have a load balancer that's connected to the public internet the load balancer can connect to these instances on the private network so what we want to do is uh we don't need to ss agents then we never need to touch these now because mutable infrastructure we're not going to update anything um so we only need to allow traffic uh on a custom tcp port there we go custom tcp port 8080 that's just because that's what i've set up for the node app uh and we're going to use a custom ip and we need the private network address for the vpc the virtual private cloud that we're currently using so if i just go to services and go to vpc i want to check the private ipcider block for the current vpc that i'm using so i'm going to go to your vpcs and if you have a custom vpc setup uh you can just check what the cider block is for that i'm using the default one which is this one and i can see that this is the private address here so i'm just going to copy this this is 172.31.0.0.16. this is the default block for the default vpcs within aws so if you haven't created a custom vpc this is what you can use so i'll put that in here so only ec2 instances on this private network can access this instance on port 8080 so that secures it down because no other network requests can come into this ec2 instance or all of the ec2 instances that i create so i'll go review and launch and i'll launch and i do not need a key pair because i'm never going to log into these things and now it will set up three ec2 instances let's go view the instances uh there's the testing one that's shut down and here are the three that i just created so these are three identical instances of the application and now we need to just distribute the load across them so while these are setting up i'm going to launch another instance um and i'm going to install nginx on it so i just chose amazon linux 2 there t2 micro um use the default bpc and i don't need any other configuration here and for the security group because this is going to uh install nginx it's going to accept http requests and then forward them on to the private instances i'm going to create a new security group i'll just call this i don't know load balancer i don't think i'll give the other one a name that's bad i do need ssh because i'm going to log in and actually install nginx on this and then i'm going to accept http request so they are http and honestly if this was in production i would set up https now i would forward http requests to port443 um so i'll just enable both of them for now but really i'll only be using hdb in this one i'm not going to set up a certificate so i'll set up that instance as well and let's go view all the instances so i should name this uh this is the load balancer and all the unnamed ones are just the different application instances um so these are now up and running and uh here is let's see if i go to the instance summary huh here's their private ip addresses so this one is 172.31.28.150 and this is a different private ip address and this has a different one too so what i want to happen is i want to set up nginx to distribute the load across those private ip addresses these cannot be accessed over the public internet so i can't enter this in my browser and actually view the app or even the public ip address because they're all private so i just got to wait okay this is now running so i'm going to use this ip address to log into this instance yes going to install nginx um and i'll put the commands in the description if you want them i'm just installing starting and enabling nginx because we're going to use that as a load balancer it's really easy to set this up as a load balancer and i'm just going to modify the nginx configuration file uh because if we come down here to the server portion of the nginx configuration uh we can delete everything below server name so all of these things just create a really small server block there we go uh and i am now going to create a new location so any requests that come in on port 80 so any http requests that come in we're going to proxy pass that to http i'm going to call this application and then what i need to do his application is just going to be a name of a block of servers that we put in here so we're just going to list uh all three ip addresses and when a request comes in it's just going to balance the load acts as a reverse proxy to each of those three different instances let's create an upstream block called it application and then here i just list the ip addresses so for that i'll need to grab these so here's one of the application instances paste that in at port 8080 and then i'll get the next one so that was really easy to implement with nginx uh now i just need to save this file and then restart nginx uh i messed up i always messed something up what did i mess up so i think i need to put server in front of each of these i really need to learn how to use them better do not need a colon that indentation all right um so let's try saving that with the server and it's still not working because i think this block actually needs to go above the server block so let's move that right here there we go anything good enough uh okay so let's try restart okay yeah so um the server the upstream block needs to go above the server outside of the server and uh each thing needs to have a server at the beginning line so i missed missed those two things so uh that's now restarted so if i curl a local host we should get a yeah oh that's awesome a different emoji each time so that means that if i go back to my instances here let's just copy the public ip address of the load balancer and i enter this into my browser i should be able to see a random emoji so this is the application working and it's going through nginx now but it's actually distributing the load between those three different instances so if i had millions of users requesting emojis uh then you know it wouldn't be my application instances that would fail because this is horizontally scaled but i've also added an endpoint here slash ec2 so if i go to that it just prints out some of the details from the ec2 instance so that we can see which instance it actually ends up on so this is the id and this is the audition all right so this is uh yeah the instance id this the the private ipv4 address so when i refresh this page it should go to a different instance so uh yeah there's a different private ip address different private ip address so it's just rotating through those three different instances it's just balancing the load between those instances so this is now horizontally scaled it's load balanced it was all very manual and tedious but this kind of idea applies when you use immutable infrastructure when you horizontally scale your application you'll just want to use a more automated way of doing it and if we now had version two of the app maybe get even more emojis whatever we would just repeat that process go create a new ec2 instance set up the vm however it needs to be for version two of this application create an image from that horizontally scale it point the load balancer at those new instances but you'd automate it so that's it for this video but stay tuned for more videos on cloud computing [Music] so i pull up inside my cup and i just sit till i can't feel it

Channel: Sam Meech-Ward

Views: 321

Keywords: load balancing, load balancer, aws tutorial, amazon web services, cloud computing, ec2 instance in aws, nginx load balancing, amazon linux 2 ami

Id: FEbfvTZCYQQ

Length: 18min 20sec (1100 seconds)

Published: Mon Sep 27 2021