Create CloudFormation Stack (EC2 + RDS) & Deploy Docker App - Episode #7

in this episode I'll show you how you can use AWS CloudFormation and AWS command-line interface the CLI to create an application stack and that consists of an ec2 instance that runs the docker for our application and an RDS instance for the my sequel and then we will deploy an API application using docker and docker compose let's take a look at the steps that we are going to go through in this video I'm going to start by creating a new key pair on AWS basically a pair of public and private keys which will be needed later to SSH on to the ec2 instances that we create next I will generate a new access key ID and a secret and then configure AWS command-line interface using the access key ID and the secret that we generated next we will create a new stack in a yeah mol file that will contain any c2 instance to start with and then test that simple stack and then update the static configuration stance docker installed on it to make the ec2 instance as the docker host and then again update the stack to add RDS with my sequel engine for our application and that completes the stack creation next I will create a docker compose file to define and configure the application service and the last step is to deploy and test so let's get started I have an account on AWS and I'm logged in let's go to the AWS management console first step is to create a new key pair so to create one on the AWS console under services go to ec2 on the ec2 dashboard locate key pairs under the networks and security settings on the left hand navigation I don't have any key pairs here so create one give it a name I will name the skipper as AWS - key one on submitting it will create a new key pair and downloads the private key which goes under the Downloads folder on my local and we can use the downloaded private key to SSH onto the ec2 instances that use the same key pair now let's go to the terminal window and create a new project directory nameless directory as deploy AWS and going to the deploy AWS directory now copy the downloaded key into this directory remember to change the file permission of this private key using chmod command set the permission as/400 to the AWS key 1.9 file now this is to ensure that the private key is not publicly viewable without which the SSH will not allow or it will just reject the private key next we will need an access key ID and a secret for the AWS command-line interface in the AWS management console under the services under security and identity section click I am the Identity and Access Management go to users section in here here we have one user we could generate an Access ID and secret for this user instead let's create a new user give this user name grant both access types programmatic access and the management console access submit and go to the next page where I'll need to grant access permissions to this user let me just check admin access for this user to make it easier review the user and permission and submit a new user has been created now and there is a new access key and a secret created for the same user now let's go back to the terminal window and configure AWS command-line interface so the command is AWS configure give this access profile a name I will name this as demo the first prompt is the access key ID now let's copy and paste the access key ID from the irritability console for the user that created and also it'll need the secret key let's copy-paste at a spa and for the default region you can pick the one that used the most so let's use us best two which is the Oregon as the default region and ignore the output format for now done AWS CLI or the command-line interface has been configured with the new profile named as demo now going forward when we need to use AWS CLI to interact with AWS account we would need to specify the profile name explicitly in the command line let's start with creating a new stack I would like to manage most of this stack creation process using AWS command-line interface instead of the AWS management console which is the web UI but we will use the management console to verify the results of the commands that we run on the terminal let's go to the cloud formation page on AWS management console here we don't have any stack created yet let's start by defining a new stack in a Yama file go to the code editor let's open the project directory deploy AWS which I created and it has the AWS q1 dot Prem file which is the private key that we copied into this directory let's create a new Yama file for our application stack definition I will name this file as stack camel but you can name this whatever you want this new application stack will consist of an ec2 instance with docker on it for our application and an RDS instance for the my sequel database the stack definition starts with the resources where each resource goes by the logical name and its definition the first resource in the stack will be the ec2 instance for the application I'll name this as app node type of this resources AWS ec2 instance followed by the properties of the ec2 instance the instance type it at a high level is the class or the size of the instance that you need for your application I will pick the smallest possible one of the cheapest one which is T 2 dot micro next property is image ID which is the ami or the Amazon machine image ID this is one of the tricky ones as the value changes by the region that you deployed the stack in let's take a look at that go to ec2 dashboard start the launch instance wizard here we have a list of AMI and for our stack with the using Ubuntu 1604 which is this AMI and here is the ami IT the one that ends with dc5 let me switch the region to a different one say North Carolina and now notice the image ID changed for Ubuntu 1604 the one that we solve for oregon ends with DC 5 and this one is different so let's copy the image ID for the Oregon which is us best to region and paste that in for the image ID value here let's cancel the launch instance wizard back to based a kml file next property to the ec2 instance is the key name this is the name of the key pair that we generated which is AWS key one now this is the basic definition of the ec2 instance but once the ec2 instance is created on AWS for us to be able to SSH on to the ec2 instance we need to have SSH port accessible on this ec2 instance along with that we also need to be able to access port 80 which is going to be our web applications port on that machine now allowing the inbound port into these ec2 instance will require us to define another resource which is going to be the security group let's create that I'll name this a resource as AB node security group in short FG and type is AWS ec2 security group like with ec2 instance this will have some properties to the security group description which is some text that describes what this security group is all about next attribute a security group ingress which will define an array or a list of ingress or inbound connection properties first one is the HTTP port which is quote number 80 and it does not weigh up to a different port so both from and to ports are the same and we will make it reachable from anywhere with the site or IP value or the source as 0 0 0 0 / 0 and the second one is for the SSH port which is 22 again both are same - and from Port and the site or IP values again same as for the HTTP port now that we have the security group defined for ec2 that has the ingress ports 4 80 and 22 let's assign this security group to the app node resource using the security groups property here I use the function ref to reference the security group which is another resource defined in the same stack now we have two resources defined in the stack one for the ec2 instance and second one is the security group for that ec2 instance the stack before adding more resources to the stack or the stack definition file now back to the terminal window we had the AWS command line interface configured already with the profile name demo let's create the sector and the command is AWS CloudFormation creates tag give a name to the stack I will name this tag as log stage provide slack definition template which is going to be stack dot EML file and the profile is demo let's also mention the region explicitly the region where we need the stack be deployed in which will be us best to or Oregon now if I don't mention the region it would pick what's configured for the demo profile as the default which is also us best too but I'll keep this explicitly mentioned in this command for now but we can change this in the later commands so everything is good here except the value of the template body the stack file path which it expect to be an absolute path with defy protocol and that will be file colon triple forward slash present working directory slash stack yamo hit enter and it triggered the new stack creation and here we have the stack ID displayed as the output of the command let's verify the stack creation on AWS management console go to the cloud formations page refresh in the stacks and we have our block stack being created here so we can track all the events progress and view resources and more here resources tab lists the two resources that be defining the stack dot ml file app node and add node security group now let's switch back to the terminal window now here we can list these stack resources using the command-line interface as well so let's take a look at that command is AWS CloudFormation describes stack resources for the blog stage stack using the profile demo let's ignore the region from the clan because it's going to pick the same one from the profile demo and that listed both the resources created along with the physical ID status timestamp number of attributes that kind of gives an idea that you can do pretty much everything using CLI and don't really have to switch back and forth between the CLI and the UI console now when the stack is complete which is complete here the physical ID feel of the ec2 instance app node turns as a link to the ec2 instance detail page open that in a new tab let's take a quick look at the instance details instance type is t2 dot micro here is the public DNS and IP address for the instance and the security group app node SG that is associated to the instance let's open the security group in another tab check the inbound section which has the configuration that we define in the app node SG ingress configs in our stack demo basically it allows TCP connections for SSH and HDPE from any source now back to the ec2 instance page copy the public dns if you prefer the shorter version you can pick the public IP address as well now in the terminal window let's SSH into the ec2 instance using the private key AWS key 1 dot them which is in the current directory so SSH is successful and I am on the ec2 instance app node but this is just the Ubuntu OS that we have here and nothing else is installed for our applications to run we need docker engine installed on this instance so let's do that as part of the slack creation process with cloud formation exit out of the NAP node let's delete the stack now so that we can create a new one with the changes to our stack definition in the stack dotty ml which would also include commands to install docker for our application to run now there is also stack update command that or we can use in some cases but in this episode I'd like to keep the scope to cover just the create and delete slack now delete the stack and the command is AWS CloudFormation delete stack name the stack is blog stage using the profile demo now this triggers the stack deletion process it's verify on the AWS console well the fact is being deleted let's update the stack template to add instructions to install docker add the user data property for the app node ec2 instance so we need to provide basics Devorah encoded script data to this property for which we would use base64 function and a pipe and in the next line we will fill in a multiple line script that would install docker make it quicker I'll paste in some script here and the first line says it's a Bosch squared or shell script and just to the instructions that install necessary packages update the package source list for docker source and install docker I think this is a pretty standard set of commands that you would run on any Ubuntu to install docker now with this updated stack definition let's create the stack again and this time the app node will have the docker installed let's verify that the previous one is deleted back to the terminal window let's run the command to create the sack again and the command is AWS CloudFormation create stack stack name is block stage using the template stacked on camel and with the profile demo let's verify this in the AWS console and the stack is being created a few minutes fast-forward we have the stack created now let's open the app node ec2 instance on a separate page and copy the public IP address once again now in the terminal window let's SSH on to ec2 instance using the public address here we are on the ec2 instance which has open to install let's check the process status and crypt for docker and we have Tocker running as well at node is now our docker host but notice by default docker listens on a unique socket which works for locally dealing with docker but we need this Liz no nor TCP port so that we can connect remotely to managed services or containers on this docker host basically we will need to alter the docker D command to add another option - edge option to add a new TCP port to listen on now let's exit out and delete these stack again while the stack is being deleted let's go and update the user data property of the ec2 instance to add some instructions or commands that would make the docker engine listen on TCP port 2375 now here I am on the documentation page of the dock or engine configuration scroll down to the configure docker section that explains the commands and instructions to follow through basically simple three steps process create this directory and add a new docker configuration file which will include the configuration for the docker engine startup command and that will be under the service section of that file XX chart we'll have minus h tcp colon colon zero zero zero zero : 23 75 this makes it to listen on 420 375 on the host let's switch back to the stack Tamil file in the editor I'll paste in about four lines of code and append that to the user data value which basically does what we just read in the documentation which is create a directory and the service configuration with the updated xxxx start command with the new - edge option to have it listen on for 2375 in the doctor configuration in that directory and then reload the daemon and restart docker service now that the act node will have the docker running and listening at port 2375 it's also update the app nodes security group and had a new ingress or inbound TCP connection for port number 23 75 copy the one for SSH and update the port to 2375 that's about it let's switch back to the terminal window and run create stakman again now this track is being created fast-forward a few minutes and we have the stack created now now go to the app node ec2 instance page copy the IP address in the terminal window let's verify that the docker is listening on port number 23 75 on the app node docker host and the command is docker - H the hosts TCP colon slash slash the IP address of the app node or the ec2 instance : port number 23 75 space PS and that returns an empty list as we don't have any containers running in that docker host now the docker setup is complete and the connection to docker host also looks good next we will need to run our application on the docker host at node now before we run our application on the App node or the docker host let's add a database to our stack we will be using RDS for my sequel let's delete the stack for now and switch back to the stack doc yamo a quick recap we have two resources defining the stack AB node 4 ec2 instance and add node security group let's add two more for DRDs I'm going to paste in the code for that which has two resources one for the database instance and the other one is for the database security group database instance ease of type AWS RDS DB instance and the properties are the database name and the database engine which is my sick master username password and instance class now this is similar to the instance type property in the app node ec2 instance I pick the smallest one again the value of allocated storage for this database of the minimum one is good the last property is the security group of the database which can take an array of security group names ours is a simple security setting and we have just one which is basically a reference to the resource which is defined below the database security group let's take a look at that database security group is of type AWS RBS DB security group not the same as the ec2 security group properties will include a description that the documentation as what this security group is all about and inbound our ingress settings now this can be a list of security group rules but a simpler version is IP range with the site riping here I have set this to this value which basically makes the RDS database instance reachable from anywhere now that completes the stack definition we have for resources defined in the stack add node AB note security group database instance for RDS and a database security group now verify that the previous stack is deleted run the create stakman again now that we have the RDS to be created as part of the stack creation it will take about 5 minutes as RDS really takes a bit longer than other resources in the stack while the stack is being created let's create a docker compose file for our application I will name this file as AB dot ml now I need an application to deploy so I'm going to be using the application that we used in episode 6 which is a rails API only application that exposes some REST API is to create show edit update blog post content that's a very simple set of API so the code for that application is on github here it is I'm going to copy the staging camel piss that in the app dot llamo and this is our compose file now let's take a quick look at that now it defines one service named as app and the image is on docker hub and here is the deposited that you might want to take a look at app service Maps port 80 - host for the container port 3000 where raised funds and it expects the my sequel database configurations be passed in environment variables now let's update these database configurations using the one that we configure in the stack dorky ml for the RDS instance DB user is log user DB name is blog password is password one two three quick compare with the stack dodgy ml that's exactly what we have configured for the RDS instance so far and for the DB host we need to wait for the stack creation to complete a few other environment variables this application expects one is the rails environment which we will set a production basically the application run as prod environment and uses the database configurations provided in the compose file another one is rails log to standard output with some value so that the application logs can be viewed as container logs then the secret key base which is something that rails application expects I believe these essays as the value is valid back in the browser let's close the tabs accept the AWS console reload the stacks and we have the new stack created now notice the stack now has four resources as defined in the stack dot llamo let's open up the database instance the RDS instance in a new tab this is the DB instance detailed view it has the database configurations here database name user and here is the database endpoint or the host name let's compute that in the docker compose file abdominal date the DB host environment variable value with this one or the new endpoint and save the file now that completes the definition of the docker compose file now we have the stack created that has an ec2 instance and a docker compose file definition for the application is also ready let's go ahead and run the application on ec2 instance now I need the IP address of the app node let's list the containers on the docker host one more time as we did before the longer syntax is darker with - edge host and the TCP IP address - 375 PS instead let me just set the environment variable duck or host with the same value as minus H option in the above command with docker host environment variable set and that the command will be docker PS without d - h option so that works and there are no containers running so let's deploy the application start with running the database migrations so that the database that is created on RDS instance has all the tables necessary for the application the command is docker compose using the compose file a piano run app with the remote option rails DB migrate now this build run rails application as a container on App no to instance with the database on RDS instance rails TV migrate is complete and let's run the application now and the command is docker compose using app dot llamo up with minus T for detached more it has started out let's verify the containers using dropper compose PS and we have one service or the container running the API application or rails API only application is running as container and it is available on port 80 on the host the app node now we should be able to access the API using either the public IP address or the domain name let's copy the public DNS or the domain name for the ec2 instance and in the rest client request the API the request type is get and the URL is HTTP the dns name that we copied port is 80 which has default slash posts now we can use both dot JSON as well submit the request or send request and it responded with 200 ok response code but with an empty response body or an empty list of blog post entries in the response let's populate some test data using rails DB seed command in the command ace docker compose using abdominal run rails DB seed now this is really for testing and normally you wouldn't do this in production environment test data has been populated now let's go into the risk line and reload the post API page and it responds with some blog post entries now let's also test one of the post detail ApS using the URL from one of the post entries in the response send request and it works - that's it we just created a simple stack that consisted of ec2 instance and an RDS instance along with necessary security groups using cloud formation for our API only application and then we use docker compose to define the application and manage and run application services on ec2 instance now before I finish up I would like to point out a few things that you can explore more about first one is the docker compose which I chose to define and run the application on the stack now docker compose is one of the many options you have if your stack is small but one can manage to do the deployment or auto deployment using cloud formation itself so that is something to explore and the second one is about the user data property in the stack dot UML the user data property be used to provide the script for the installation of the stud that we need it on ec2 instance such as the docker or configurational docker that's an easy approach but there are better options with confirmation that you might want explore the last but the most important one is the docker service or the docker engine that we configured to listen on port 2375 now except for testing this definitely is not a good idea as the docker host in this case does not require any authentication and communication is not encrypted instead docker engine should be TLS enabled with the SSL certificate now that solves both authentication and encryption problem now that's pretty much for this episode thanks for watching

Channel: Chandra Shettigar

Views: 41,342

Rating: 4.9820361 out of 5

Keywords: AWS, CloudFormation, Docker, DockerCompose, REST API, Ruby on Rails, EC2, RDS, MySQL, IAM, AWS CLI, Deploy Docker, Docker on AWS, Stack, aws cloudformation, cloudformation tutorial, cloud formation, cloud formation aws, aws cloud formation, cloudformation aws, cloudformation docker, aws cloudformation tutorial, cloudformation ec2, create cloudformation stack, cloud formation template aws, cloudformation rds, cloud formation tutorial, aws cloudformation docker, cloudformation yaml

Id: 8J0g_xWUzV0

Length: 24min 58sec (1498 seconds)

Published: Sun Apr 23 2017