Infrastructure as Code - Crash Course

in this course you will learn all about infrastructurist code i'll start by giving you a general overview of the different concepts and then i'll show you how to implement infrastructure as code with three different labs this course will use python and aws but the concepts will apply to other languages and cloud providers so let's start by talking about what infrastructure as code is to put it simply it's setting up your infrastructure as code by infrastructure i mean all the different things needed to deploy your software into a cloud environment that can mean things like virtual machines containers or serverless functions infrastructure also means all the other pieces of infrastructure you need to set up to make that successful that can be security like aim and kms or networking or some of the monitoring and logging capabilities you can also use code to configure and set up data stores these are the things that your application needs to store and manage data the last piece of infrastructure landscape is the applications themselves and getting the applications that we're building into the infrastructure all of these different pieces of infrastructure can be set up using code software as a service or sas is related to infrastructure's code software as a service allows users to connect to and use cloud-based apps over the internet sas provides a complete software solution that you purchase on a pay-as-you-go basis from a cloud service provider you rent the use of an app or software for yourself or for your organization from a cloud service provider and connect to it over the internet common examples are email office tools crms and development tools infrastructure as a service is part of software as a service infrastructure as a service is a cloud computing service where companies rent servers for compute and storage in the cloud this allows users to run operating systems or applications on the servers without the maintenance and operating costs of those servers so infrastructure as code involves using code to manage your infrastructure as a service it's becoming more and more important to automate infrastructure because applications can be deployed to production up to 100 times per day and you don't want to be doing that all manually it's also helpful for infrastructure to be provisioned or d provisioned in response to load we need to answer questions such as how do we get the images for our vms how do we get the container images in how do we package up and ship code into serverless functions over the years there has been a transition with how people are using cloud infrastructure within their organization in the first wave it was fairly simple the infrastructure were fairly static it was often single virtual machines that you just accessed through ssh it got a little more complex in the second wave there were more containers and people started using provisioning tools to specify the application behaviors people use things like docker and datadog modern cloud infrastructure has added way more complexity it uses containers serverless and more managed services as part of the applications there are now way more different pieces involved in how people build the infrastructure infrastructure as code is becoming a more important part of how people build and deliver applications because infrastructure as code is what describes the glue between all the different edges on these diagrams this diagram for the modern infrastructure may look more complicated than the previous ones but it can actually be easier to maintain a key benefit is that the dark gray squares in the diagram are the only part which is the code you own and that part is smaller than the previous ways of doing things so a lot of the operational burden has been decreased compared to how it used to be done with infrastructure as code it is now easier to build deploy and manage infrastructure in a repeatable and scalable way on a variety of cloud service providers infrastructure as code makes it easier to manage that process alongside your actual application code now there is a greater focus on the glue between the different services that are being managed by a cloud provider and these services can be deployed in a repeatable and scalable way infrastructure code takes on an increasingly more important role in how you manage everything in the modern cloud so here are the three main ways that can be used to manage all the resources manual that's how it used to always be done you just point and click to create or modify resources in the console you can do it like through the aws console you can just point and click to what things you want to create then there's ad hoc automation this is when you use cli commands or scripts to create and modify resources so you're kind of using some code but it's not going to be able to be reused over and over and modified very easily the third way is what we're talking about infrastructure as code there's provisioning which is declaratively creating and modifying resources and configuration which means to change the state of an existing resource post provisioning infrastructure as code gives us the ability to write down what we want the desired state of our infrastructure to be i'll be showing you how to do some of these things later there are a few different ways to write infrastructures code so one approach is to use json here's an example you can see this is a way to tell aws what resources you want to have you can see the resources the ec2 instance the instant security group and just everything you're going to need just in a json format another method is to use domain specific language or dsl this is a custom method specific to the tool or cloud provider you are using so here's an example you can see it does kind of look like code but it's no no it's not like a popular programming language or even the exact way that json usually is written yet another way of defining infrastructure using code is to use a well-known programming language for instance polumi can be used to write infrastructure as code using typescript javascript python go and net here's an example using typescript and later we'll be using python using code gives the ability to do things that are not possible in some of the other methods in this example there's a for loop that creates an instance for each availability zone code gives the ability to use loops conditionals classes packages and more using popular programming languages also allows the use of common ides linters and test frameworks so let's get into some real world examples we're about to set up some actual infrastructure as code there are a few different services that allow you to use popular languages to create infrastructure as code in this course we'll be using pollumi this course was made possible through a grant from blooming pollumi is an open source infrastructure as code tool for creating deploying and managing cloud infrastructure pollumi works with traditional infrastructure like vms networks and databases in addition to modern architectures including containers kubernetes clusters and serverless functions plumi supports dozens of cloud service providers we'll be using python and deploying on aws though it can be done with other programming languages and cloud providers we'll start with a simple example this will show an end-to-end experience of working with plumi using very simple resources in this first example we'll do the following create a new project configure aws provision infrastructure update infrastructure make your stack configurable create a second stack and destroy the infrastructure we'll use an s3 bucket and then work through the life cycle with a simple set of resources then in future examples you will learn how to implement more complex things with pollumi infrastructure is organized into projects each project is a single program that when run declares the desired infrastructure for pollumi to manage before we start the first lab make sure you have polomi installed the way to install is different depending on your operating system you can check the link in the description for all the instructions for how to install different operating systems one way if you have home brew is just brew install pollumi like i mentioned we'll also be using aws you have to make sure you have an aws account and have the cli set up and authenticated i have a link in the description that shows you how to sign up for a free aws account also there's another link that shows how to install the aws cli for your operating system from aws you're going to need an access key id and a secret access key let me show you how to get that i'm at this url right here um you can do without the v2 at the end just the console.aws.amazon.com iam and then i'm going to go to users here and then i'm going to add a user i'm going to make a new user just for this project and the user is just going to be infrastructure as code or iac and i'm going to click that i want to get the access key here and next permissions i'll just add administrator access so to make sure it can do whatever needs to do and then i'll just go through just keep clicking clicking next and then create user okay now we have the amazon access key and the secret access key so you're going to actually have to copy the access key and the secret access key from this page and you can get that obviously by clicking show to show the secret access key let me show you where you're going to need those things go over to your command line and type in aws configure okay now this is where you're going to enter your access key i'm just going to copy it from here and then the aws the aws secret access key now yours probably won't have this here because i've already done that so it's seeing if i want to use what i already have but i want to put in a brand new one here and then we can just click none for the next few things here okay aws is set up so the first step to starting a pollumi project is to create a new directory each pollumi project lives in its own directory so i'm going to create one first i'm going to go into projects okay i'm gonna make a directory and i'm just gonna call it iac lab one now just jump into my directory i just created okay i'll do paloomi new python because we're making a python project and we'll just select yes to everything now i've already used pollumi on this computer but if this is the first time you've used pollumi you will be directed to enter an access code or log in and it will tell you the website to go to get to get the access code but it's app.paloomy.com account slash tokens you will have to create a plumi account and then you will be able to get your access code so this command has created all the files we need it's initialized a new stack named dev which is an instance of our project we now need to install our dependencies as part of our virtual environment so let's set up our virtual environment we're going to create a virtual environment and install the required python packages the python module used to create and manage virtual environments is called v e and v so let's use these commands first we'll do python 3 m v e and v v e n v and then source v e and v slash been slash activate okay i've activated my virtual environment and now i'm going to install the requirements pip 3 install requirements dot txt that requirements.txt file was created when we initialized our polumi project now let's open this up i'm going to open up in visual studio code but you can use any code editor you want okay these are all the files that got installed when we did their pluminew this main.pie is the program's main entry point file this is the file that we'll be putting all of our code in now we have the requirements.txt that's just your project's pip dependency information and then we have pollumi.yaml which is the project's metadata containing its name and language you can see we have python and then we have the name here and the virtual environment so if we go back into the main.pi we'll notice that we just have one line of code that's just automatically put in there which is import gloomy now there could be more or less code in here depending on which way we run the plumi new command but we just wrote ran it with plummy new python so it just started with import polymer which is the line of python so now that we have the basic project let's configure aws support for it i'm actually going to open the terminal right from within vs code okay i'm going to install something so pip 3 install pollumi aws now i'm going to make our code take up more of the room here and i'm going to add another line so import i'll import we just installed so import pollumi and actually recommend it right here as aws okay now we're going to configure an aws region which is whatever region you want to deploy to by we're going to run something in the command line so i'll go back down to the command line and i'm going to do pollumi config set aws region and you can use any aws region that you like but in my case i'm going to use us east one okay now we're actually going to start doing some infrastructure as code we're going to provision infrastructure so now that we have a project configured to use adws we'll create some basic infrastructure we'll start with a simple s3 bucket so let's declare a new bucket right up in our python file here i'm going to say bucket equals a aws.s3 dot well s3 dot bucket bucket and then you can call anything you want but i'm going to call it my bucket okay i'm going to save this and we're going to preview the changes so i'm just going to go back to my terminal down here and type in pollu me up so pluto me up is the command you'll be running a lot with palumi which is going to basically run our main.pi file and set up our infrastructure how we configured it so it looks like there's an error here so let's see what that is hmm looks like there was a mistake with my aws login let me just try doing the login again and see if that fixes the problem aws configure now i'm going to get my access key now try running pollumi up again okay work this time so let me bring up this we can see more of what the output was here so this is a preview the command evaluates your program determines the resource updates to make and shows us an outline of these changes so this is what's going to happen we're going to create this stack and we're going to create this bucket here and it hasn't done anything yet we can decide what we want to do for now i'm going to go to details to see more details so you can see now we can see that the acl is private you can see the actual bucket name we call that my bucket but plume is going to add this at the end so we can actually create a bunch of buckets with the same name but it's just going to have a different thing at the end here and you can just see some extra details about this and i do want to perform this update so i'll just go to yes and okay return so now our s3 bucket has been created in our aws account so if we view our buckets on the aws website you'll see the new bucket so i'm going to go to aws and then go to the s3 section and it's going to show this bucket we just created so it's currently empty but we can see the bucket that we just created with pallumi and there's also this url here if we go to this url i'll let's do uh i'll just do command click and i'm going to have to log into plumi to get this to work out so now we're on the plumi console which records your deployment history so right now we've just done one one thing so you can see one thing on here okay next i'm going to show you how to export your new bucket name so we're going to go back into the code area and to inspect your new bucket you'll need its physical aws name well we did see we were able to see the name up here when we went to more details but there's another way to get the name that i will show you using the code so this is called the logical name my bucket but like i said before the aws name will be a little different programs can export variables which will be shown in the cli and recorded for each deployment so let me show you how to export the bucket's name by adding a line of code in here so right after we get the bucket here i'm going to do pollumi.export x export and then i'm going to export the bucket name so that's just what we're calling this value which we're going to get as bucket.bucket so this bucket.bucket will be the bucket name so let me just save this and then we'll go back into the terminal and i'll do pollumi up and now we can see there's a new line here outputs so you can in your python code you can export certain information that will show up in the output section when you do blue me up so i'll just select yes to perform the update and you can see it's unchanged because the only update was just to output the bucket name here you can see all the output by running this command in the terminal pollumi stack output you can see it's there's only one output the bucket name and then it has the value of it right here now we can run the aws cli to list the objects in this new bucket and we can get the bucket name using the command from above let me show you how you do that so we're going to do aws s3 ls and then i'm going to run that command right in here pollumi stack output bucket name so that's going to get the bucket name and if we run that command nothing's going to really happen because there's nothing in the bucket so it's returning what's in the bucket but there's nothing in the bucket so there's nothing to be returned but we'll change that shortly we're going to now update the infrastructure we just saw how to create new infrastructure from scratch so now we're going to make a few updates we're going to add an object to the bucket we're going to serve content from the bucket as a website and we're going to pragmatically create infrastructure this will demonstrate how declarative infrastructure as code tools can be used not just for initial provisioning but also subsequent changes to existing resources so we're going to add an object to the bucket i am going to create a new directory i'm going to just go to new folder i could click this button up here this folder is going to be called site and then i'm going to make a new file and this will be called index.html okay i'm just going to make an extremely simple website here html and body and then just an h1 tag and it's going to say hello everybody okay now let me rearrange things here we'll go back to our main.pi and let's add some lines of code so we can add that file to our bucket so i'm going to add import and put a new library the os library so we can access uh stuff from our operating system the file path in particular so let's see actually we'll put it right here file path equals os dot path dot join and then we'll put the directory name site and then the file name index.html now we're going to create an object variable and we'll do aws.s3 dot bucket object index.html so the object is index.html and then the bucket is going to be dot bucket and that's how you get the name of the bucket that we created and for source i'll put polumi file asset file path so this is how you use polumi to get this asset this file path right here we're going to create as the object that goes into our bucket so i'm going to save that and enter terminal we'll do plumi up and then i'll just look at this more oh not fire asset file asset because this is a file asset so i'll go to plume up see that makes a lot more sense to you doesn't it than fire asset okay so you can see it's showing what the change is going to be we're going to create this object which is the index.html we're going to create that so i will go to yes to do these changes perform this update so a single resource is added and the two existing resources are left unchanged this is a key attribute of infrastructure as code such tools determine the minimal set of changes necessary to update your infrastructure from one change to the next so let's list again the contents the contents of the bucket i'll just press up a few times we'll use the same line from before and now let's see what's in the bucket it's showing index.html and if i go back over to my web browser and if i refresh my page that shows the bucket on aws we should see yep we see this index.html right here next step we're going to serve content from the bucket as a website to do that we'll need to update a few properties first the bucket needs a website property that sets the default index document the index.html that can that can be achieved by making a few changes to our python file we have our bucket right here and i'm going to after the buck and nam and add a comma and then i'm going to put website equals and then we just have to set the index document and we'll just set that to index.html and make sure that's in a string here okay next the index.html object will need two changes so let's go down here to see this index.html object that's right here we're going to need an acl which is an access control list of public read so right after the source i'll put acl equals public read there now you can see that better this makes it so it can be accessed anonymously over the internet we also need a content type so it's served as html so for the content type to work let's go back up to the top we're going to import something import mime types because we're going to set the mime type and then down here after the file path we're going to get the mime type of the file so mime type and what we'll put it underscore because this is going to return two things but we only need one of the things which is the mime type so we'll do mime types dot guess type and we'll guess the type of the file path and now down here we can set the content type to the mime type so now we have the content type so it's served as html we'll also need to export the resulting buckets endpoint url so we can easily access it so let me show you how you would do that so here we're exporting the bucket name and now let's export the bucket endpoint the full url so we'll do pollumi dot export and this will be called the bucket endpoint we're going to use this concat function from polumi to concat a string and something else so this bucket.bucket it's actually an object it's not a string but it does return a string so if we want to combine something like this with a string we have to use uh concat let me show you how that's done we're going to do pollumi dot output dot concat and then i'm going to put the two things we're going to concat the first thing is going to be a string https came and see what i'm writing here let's see if i can http let's do http colon slash slash and then we'll get a comma here and then just bucket dot web site endpoint so we're just concatenate we're concatenating the http and this website endpoint okay let's deploy the changes so well let me save it and then we'll do pollumi up if i go up here we can see it's making some updates and i'll go to yes okay now we have this url here so we can use this to access that index.html file so i can actually just copy this url stored on amazon aws and now i can open up the url right in my web browser it says hello everybody i can also use curl to get it so back in my terminal do curl and now i am going to run a command right here to get the website url lumi stack output and then you can get any of the things that get output and we'll get bucket endpoint okay see we got this here's the text of our html file next we are going to make the stack configurable right now the bucket's contents are hard coded we're going to make the location of the contents configurable and add support for populating the bucket with an entire directory's worth of contents so the next step is to add a config variable right now we hard code the directory of the site let me show you here so we have we're hard coding this directory here of the site so we're going to configure it to make it easy to change the location without editing the program so right below the imports here i'm gonna do config equals pollumi dot config that's going to make it possible to access some config settings that we set up and we're going to specifically access the site directory and we're going to get it from config dot require and then we're going to get the site dir and i remembered i see that i spelled paloomi wrong there we go and we are going to set this in a minute the site directory but first let's update our program to use this citer variable so instead of getting the site here i'm just going to paste in the variable name okay so now what we're going to do i'm going to save this and then i'm going to go back into my files i'm going to rename this site directory i'm going to rename it www now you may not actually do this in like a real life situation but we're just doing this as an example to see how this can be helpful so let me go into my terminal and i'm gonna do pollumi up okay it's saying air missing required configuration variable this site dur variable and it says please set a value using the command plume config set ic lab 1 site directory with the value so that's what we're going to do now i'm just going to copy this command here and then i'm just going to paste it in and then i'm going to set it to www so now the directory you'll be looking for is the www directory instead of the site directory okay i'll just do plume up again and i will perform the update okay that works so now we don't have to have the the directory hard hard-coded in here we can use these config variables and you can see how you'll be able to use that with other types of data as well okay now we're going to add more files instead of hard coding the set of files which we do now let me show you what i mean right now we hard code that we're adding the index.html file to our bucket but instead of doing that we're going to change the program to read the entire contents of the www directory and add every file from that directory to our bucket so let's start by adding a new file to that directory so i'm going to go into the directory and click the new file button and this is going to be called about.html and this is just going to be a simple thing you can make it anything you want i'm just going to make an extremely simple file with a paragraph it will say it's about so i want to say something about me and i am not a cat so that's something about me okay let's go back to the main.pi file and we're going to replace the object allocation code and now we are going to update the object allocation code so this right here we're going to put all of this into a for loop so i'm going to say for file in os dot list directory site directory and then i'm gonna just indent all this so it's all in that directory and see we have the the file here so now instead of index.html i'm going to put this variable file so now we are using this for loop to go through every file and adding all those files to our bucket now this is one of the really cool things about infrastructure as code and using an actual programming language like python is that you can use for loops and that can make things go a lot that can make things a lot simpler and save a lot of time like we didn't have to use a for loop we could have just copied and pasted this code a bunch of times and and then added each file individually in insta instead of using this for loop but this is going to save us a lot of time especially if we have tons of files that we want to put into our bucket so let's go back to the terminal i'll do actually have to save first here and then i'll do pollumi up so you can see what's going to do it's going to create this file this is about.html and let's do that i'll go to yes now i'm back in my web browser i'm going to put a slash about dot html at the end and now we can see i'm not a cat i can zoom in on that so you can see a little better it's just a paragraph tag so it's pretty small we can also access it through curl the same ways before let me just press up in here but i'll just put slash about.html at the end and then yep here it is here's the about.html file we created now we're going to create a second stack it's easy to create multiple instances of the same project in pollumi each instance is called a stack this is useful if you have multiple development or test environments like staging versus production or if you want to scale a given infrastructure across many regions so let me show you how to create and configure a new stack we'll create a stack named prod so i'm still in the terminal down here i'll do pollumi stack and knit prod and i just created it now next i'll configure the two required variables the same ones we did before so we'll do paloomy config set this is going to be the region aws region and we're going to do it different from before this time it's going to be eu west one you can basically set this to any aws region we'll also set the the site directory paloomi config set iec lab one site directory and instead of the www this is gonna be www.prod i'm just showing where you can have different site directories for different projects for different stacks so let's see the list of stacks we have paloomi stack ls we have the dev stack and we have the prod or production stack now let's populate the new site directory we could have used the existing www directory for the site directory but for this example we'll use the the www prod directory to demonstrate how it can be configured we will make a directory wwe prod and then i'm just going to create a file right from here we could have created the directory from here so where's our new directory right here i'll just create a new file and it's just going to be another index.html file and we'll just make it pretty similar let's copy this other index.html file but this time we're going to add a paragraph tag here and put in production so then we will be able to know the difference between the production one and the development one so just save that and then we'll just deploy the changes plume up we don't need that up anymore it says we need to set the command the configuration variable apparently this got set wrong oh it's because i call the site i made a mistake a long time ago where's the side drill which should have been site der so let's just go back over here and let's see this is supposed to be site der and i must of uh yeah so now let's try this pollumi up and then i'm actually going to also have to create the site the site der configuration variable for the development stack but we're on the this we're on the production stack now so let's keep going with this okay so we can see the new buckets new objects it's creating and we can go to details just like before and you can see additional information but i will just go to yes and perform the update okay let's try going to this url here i click command click but i'm just going to copy it and then i'm going to go to my you website here hello everybody in production so we're now seeing the production website we can also curl it just like before it's actually pretty much the same command so if i just press up a few times and we see hello everybody in production and especially when you're practicing whenever you get to the end of what you're doing you want to destroy everything destroy the resources this is especially important in aws you don't want to accidentally be charged something if you have stuff on aws for a long period of time so to destroy all the resources we can just use pollumi destroy and that will make sure everything gets destroyed just like before it's going to show us a preview and it's showing that it's going to be deleting all these things three to delete and yes i want to do that and you'll see this message on the bottom the resource and the stack may have been deleted but the history and configuration associated with the stack are still maintained if you want to remove the stack completely run pollumi stack remove fraud which we do want to do so paloomi stack remove prod and i'll just type in prod to remove that but remember we have another stack we have the dev stack so we're gonna have to switch back to the dev stack using pollumi stack select dev and then once we've went back to that stack we can do halloumi destroy and then i will say yes we want to destroy that and then we want to run this other command again gloomy stack remove dev and type in dev okay let's just verify that the stacks are gone so i'll do pollumi stack list and there are no stacks okay we just completed the first lab the next few labs are a little shorter and demonstrate some more advanced tasks the second lab is all about provisioning ec2 virtual machines amazon elastic cloud compute also known as amazon ec2 is a web service that provides secure resizable compute capacity in the cloud it's designed to make web scale cloud computing easier for developers amazon ec2 simple web service interface allows you to obtain and configure capacity with minimal friction so in this lab you'll first create a single ec2 virtual machine afterwards we'll scale that out to a vm per availability zone in your region and then add a load balancer to spread load across the entire fleet now we could do it all within the directory we already created and just update our python file just remove everything from our python file and start from scratch but we'll create a new one to get more practice we'll create a new directory to get more practice so let me create a new directory called iac lab 2 and i'll go right into it cd iac lab 2 and then i'll just open that with my code editor in this case visual studio code and before i initialize the project in the terminal and then switch over to code editor now i'll just initialize the project right from the terminal within my code editor so let me open up the terminal and we'll do pollumi new aws python so this is a little different from before why now we just have to make sure our virtual environment is set up so do python 3 then then and then source then then activate let's try putting an e at the end that works better okay now let's install the requirements and let me configure the aws region so plumi config set aws region us west two and again you can use any region for this okay you can see the files here are pretty much the same before we have the requirements we have the yaml file move this down in the main.pi you can see this is a little different we've already initialized this with creating a bucket and exporting the bucket you can see the way we created this with the plumey new already added some some extra code creating an s3 bucket i'm going to do this a little differently so i'm actually going to delete all this and i'm going to import pollumi as aws now we're going to dynamically query the amazon linux machine we'll do this in code and so that will avoid needing to hard code the machine image in other words it's ami ami is the amazon machine image and that provides information required to launch an instance so let's get the ami with code so ami equals aws dot ec 2 dot get ami and we want to get the most recent one okay you can barely see that okay mo so most recent we can set that to true and then i'm actually just going to paste in some stuff here everything's in the description all the code and everything there's a link to an article that will have all this code but this allows us to set the owner of the ami and some filters we want to use so we also need to grab the default virtual private cloud which is a service that lets you launch aws resources in a logically isolated virtual network that you defined that's available in our aws account so to get that virtual private cloud we'll just do vpc equals aws.ec2.getv and then we'll just set the default to equal true now we'll create an aws security group this will enable ping over icmp and http traffic on port 80. so for this i will just paste it in and we can see this is how you would create a security group on here and again you can get that code right from the article in the description now we're going to create a server and this server is actually going to have a startup script that spins up a simple python web server i'm just going to paste in some code and then i'll explain it so we're creating an instance of the server we're creating a web server and then this is the type a t2.micro and this comes right from aws the types of different servers that you can create so you could do a t3 or t1 and there's just different types you can do and then we're getting the security group ids right from what we already created up here the group ids and then the ami is from what we already created and then this is an interesting thing the user data here we are creating a simple python web server now for most real world applications you would want to create a dedicated image for your application rather than embedding the script in the code like this but for this example we're just trying to spin up something really quick okay now we are going to export the ec2 instances resulting i p address and host name so we'll do pollumi dot export and then we're going to export the ip and then server dot public ip we're getting the ip is coming directly from aws here and then the host name which in this case is the public dns okay we're going to provision the ec2 instance and access it so to provision it we'll do pollute me up wait first save save and then pollute me up and then let's see what it shows looks like there's an air i think we made a minor mistake here oh api that should be ami so let's save that and then do blue me up again okay and you can see it's showing what it's about to create and i'll just go into yes okay if we go up we can see we have the host name and the ip address so i'm actually just going to take this hostname and copy it we're going to go right into our web browser so if i paste it in here okay it took a few seconds to start working but it must have finished provisioning so it says hello world and another way to do is write from the command line interestingly it looks like a new version of polumi must have came out in the middle of me making this tutorial because it wasn't saying that before okay but right now i'm going to do this curl and do pull pull me stack output hostname and then we can see the hello world right here and i might as well just install the new version of pollumi while i'm at it okay that was simple so the next thing we're going to do is add more ec2 instances we're going to create multiple instances that are each running the same python web server and we're going to make them across all aws availability zones in the region so let's go back up to our code and the part of the code that creates this web server and this instance and it exports resulting ip address and host name we're going to modify this a bit well first of all we're going to make a new there's going to be a new ip and host name for each aws availability zone so let's store those in in an array or a list so ips equals and then we're just making an empty list that we're going to be filling host names equals and then we're going to put this into a for loop because we're going to make an instance for each element for each availability zone so for a z and aws dot get of the availability zone we can get right here z dot names so for each availability zone now what i'm going to do i'm going to put this into the for loop and then we'll just do some minor updates so this will be different for each availability zone so for where it says web server i'm going to use the python f string to put a variable in here dash a z which is the availability zone and then for the well we'll add what the availability zone is so i'll put availability zone and it's just going to be a z and now we are going to make it so each and now we're going to make it so it says hello world we're going to make it so this is different the index.html files can be different for each availability zone so for that to work let me just add a few characters here and then we'll just do dot format okay so now it's going to fill in the availability zone uh for in this spot here in the index.html file and we'll go down here and the ip is going to be ips and then we'll use that variable and actually we didn't fill that variable yet so we still have to do that so host names host names and then right here still in the for loop ips.append server dot public ip and then host names append server dot public dns okay so now we'll be exporting the list of ips in the list of host names so let's do another plug me up i'll save that and we'll do pollumi up okay so we're going to be deleting our original one and creating four new ones if we go up here so there's there's four of them for each availability zone and you can see them right here west 2a 2b b2c and 2d so i'll perform that update okay it looks like we have an error the requested instance type which is t2.micro is not supported in the availability zone but just this one so i'm going to copy this and we're going to use an if statement to make sure we create an instance everywhere except that availability zone so if not a z equals us west d2 then we'll create the server okay just let me just indent all this here and then i'll save it pull me up i'll go yes okay now we have these three outputs so i can just try going to one in the web browser and you can see this one is in 2a let's try another one and this one's 2b so each web server is going to include in the response what availability zone it's in but this isn't very realistic in the real world you're not going to want all these different urls for each availability zone we're going to now add a load balancer the load balancer is going to be able to distribute the load evenly we'll just have a single url that we can go to and the load balancer will automatically determine which server or which availability zone to get the data from this is the way you can handle a lot of people going to a website at once and the website can be loaded from different places depending on where the load is the lowest so it can kind of balance the load across all the servers okay back over to our code we need to add an egress rule to our security group whenever you add a listener to your load balancer or update the health check port for a target group used by the load balancer to route requests you must verify that the security groups associated with the load balancer allow traffic to the new port in both directions so to do that i'm just going to paste in the egress here this is required to ensure the security group ingress rules don't conflict with the load balancers so now we'll define the alb let me go right after this security group definition i'm going to add the load balancer creation steps someone just paste in some code here and i'll go over it so we already have we already got the vpc up here but we also need the vpc subnets and then we're going to create the load balancer like this we're gonna it's internal equals false so it can be accessed by the public we're going to use the same the group id for the same security groups and then the subnets we get right up here and then we're creating a target group here and then a listener so we've defined the alb it's target group and some listeners but we haven't actually added the ec2 instance to the alb so this is how we're going to add the instance let's go down to our creation of the ec2 the ec2 creation was update this to the security groups and then we'll attach the instances so we'll do attachment equals aws.lb dot target group attachment and then this f string here target group are in is going to be the target group dot arn and the target id is going to be the server dot private ip and then it's going to be on port 80. okay we just want to export one more thing the url and this is going to be the main url we're going to be using which it has the load balancer okay this is all the infrastructure we need for our load balanced web server so i'll just save that and we'll apply it with polumi up and i'll perform the update okay here is the load balancer url if i just copy this and i'm going to do a curl to it now watch i'm just going to keep doing it a few times and every time i do this curl you can see that a different url is loaded because c says us west 2a 2c 2b 2b 2a so the load balancer is working i have one url but each time i go to the url it's being loaded from a different server because the load is being distributed so that works that's how you create a load balancer with infrastructure as code since we're done with that i'm now going to destroy everything since this is just a test i don't want these web servers still going on aws so to destroy everything i'll just do paloomy destroy and then i have to see do i want to delete all this stuff yes i do okay just like it says here we can also run this command plume stack remove dev to remove the stack completely in this final lab we'll use pollumi to deploy a docker image to ecs with fargate so let's start from the beginning again to make more practice so let me go out of this directory and i'm going to create a new directory and then i'll just change into it and i will initialize the project okay we'll do pollumi new aws python and then i'll just fill this in or just hit enter to use the default so just like before it's creating all the files we need it's initializing a new stack named dev which is an instance of our project and it's installing the needed package dependencies from pi pi let's open up this in vs code okay here's the project in nvs code the same files from before we have the main.pi which is going to be our main file where we put all the python code like before and you can see down in the terminal we're in our virtual environment and let's configure the aws region now it did do that when we were setting things up but just to make sure we have the right region pollumi config set aws region us west 2 and then we'll go back up to our code and i'm actually just going to delete a lot of this because we're going to start from scratch we're going to start by creating creating an ecs cluster so we imported pollumi now let's import plumy aws as aws and now we'll create a new cluster we can use the default values so it's going to be pretty concise so cluster equals aws.ecs that cluster cluster now we're going to create a load balance container service so we're going to allocate the application load balancer or the alb and listen for http traffic port 80. in order to do this we'll need to find the default vpc and the subset groups for it okay i'll just paste in some code here and i'll just look through it i'm going to go through it pretty quick a lot of it's similar to the last lab we did so we create the vpc and the vpc subnets we got the security group here there's the egress and then we got the load balancer and the target group and the listener so i'll save that and run plummy up and then i'll make the update so we've fleshed out our infrastructure and added a load balancer that we can add infrastructure to in the next step we'll run a container aws fargate is a service that enables a user to run containers on amazon's cloud computing platform without the need to manage the underlying infrastructure in order to create a fargate service we'll need to add an iam role and a task definition and service the ecs cluster will run the nginx image from the docker hub so we're not actually going to create our own docker image for this example we're just getting the image right from docker hub so first we'll need to add an import at the top of our file go to the top and put import json now at the bottom i'm going to paste in some more code here again it's all in the description if you want to get the code so here we define our iam role and attach a policy so this section right here we're defining the role here and then we're going to attach this policy then we define a task definition for our ecs service you can see the image is the nginx image and then here is the service and then at the very end we export the url we get from here the alb the dns name so let's provision the cluster and service we'll i'll save and then do pollumi up this is going to output the status and resulting load balancer url perform the update so if i pull this up we can see the status and the resulting load balancer url i'm going to copy this url and we're actually going to try going to it in the web browser so if i load this it says service temporary on the temporarily unavailable but let's see if i refresh it a few times okay keep refreshing it eventually says welcome to nginx we got the web server says if you see this place the nginx web server is successfully installed and working we can also go to the terminal and do this curl plume stack output url so it's just going to actually just curl that url and you can see we have the same html that we saw in our web browser that the nginx web server successfully installed so now let's update the service we're going to change the desired count from 1 to 3. so now we are making three containers save that and then i'll do pull me up and then i'll say i want to update i'm over on the aws console and if i go to the ecs clusters page and i make sure i have the right region in here the one that i set up for my on polumi we can see the cluster that i created we have one service it's running two tasks so desired test three and it's running two so we can see the desired count up here is three just like we set up in pollumi okay we're actually done with this so we're going to destroy everything pollumi destroy and yes okay then we'll remove the stack completely with pollumi remove dev okay we're done with the third lab thanks for watching i hope this spurred your interest to start trying to use some infrastructurized code on some of your own projects

Channel: freeCodeCamp.org

Views: 86,171

Keywords:

Id: EtEb40LE5zQ

Length: 69min 41sec (4181 seconds)

Published: Tue Nov 16 2021