🚀(English) Mastering Terraform: ECS Service Discovery & Connect Tutorial Infrastructure & Deploy💻P1

assalam alikum my name is Muhammad Asim I'm working as devops engineer guys in this video I'm going to show you how we can set up the ECS cluster which is the elastic um container service uh AWS and we are going to set up that through terraform and we're going to learn uh the ECS service Discovery right and also we are going to uh see some of the debugging stuff that shows um usually devops or developer phase uh regarding the ECS right so guys what I've done I've created a repository for you guys and in this repository I've set up the terraform code we are going to provision uh all interest uh structure through step by step right and let me navigate to my channel and if you just type ECS you see in past uh I've created this uh video regarding the ECS service Discovery and now we are going to set up pretty much the same stuff but some of the stuff which we are going to cover uh is the new uh stuff like The Innovation from the aw side um which is the uh aw service connect uh I've added in the terraform code if you want to know what is the um AWS service connect simply navigate to this official video and in this video space everything they have described regarding that particular service right so this is the official AWS Channel and you can learn about that so I'm not going to cover which has already been covered in detail um by AWS and right so you can see that particular video if you want to see the detail what um AWS uh service connect is right so in uh General it is uh more of a additional functionality uh which provides regarding the you know uh rres in your service if you want to do that and you know give you lot more for instance if you use H in communities so that sort of functionality U is now provided by AWS right so anyway now what we are going to do we are going to provision our uh infrastructure in AWS so already logged in into my sandbox environment guys what you need to do uh you need to clone this repos let me do that so instead of uh SSH you can do https because this is the public repository so and I own this repository that's why I'm using SS right so what I'm going to do I'm going to take a shell into my running container so let me do that and in that container I've already logged in with my credentials so if I do doer PS This is my container right so I'm going to take a shell into that and I have all the stuff um already ready in this uh in this particular shell I mean in this particular container right so what I'm going to do um in fact I've already clone this reper right so uh you need to Simply clone this and I'm going to open this in my vs guys uh the first step is that we are going to set up the network right so excuse me so let me clone this right and what I'm going to do let me show you this part um we are going to first of all provision the network so first of all uh let me open this in another tab as well so just a minute we are going to cover also the uh vme as well so in order to set up the back end for our uh infrastructure through AWS we are going to create this particular bucket so you need to Simply copy this particular command and paste uh in your shell you need to make sure you logged in with the uh access keys and uh secr access keys I mean the pratic access should be available in your account right so the bucket is created and what I'm going to do I'm going to enable the versioning in that particular bucket so let me do that sorry paste it again and the versioning is enabled so if I navigate uh to my S3 bucket we'll see the one bucket is going to be available right and in that particular bucket we are going to uh provision our infrastructure P so this is the bucket which we are going to use right and currently we don't have any infrastructure so guys what I'm going to do I'm going to I'm going to navigate to VPC section and I'm going to do dataform in it terraform in it and to terraform apply Dash Auto Dash approve so guys what it does it is going to download uh this uh remote uh PPC um module right and it is going to then apply that so I'm going to cover the code now regarding the bpc so guys this is our back end so let me tell you about the backend this is the bucket we have already set up through our aw CLI right and in that particular key we are going to create uh um directory uh with the name terraform in that terraform we will uh add our vbc directory in that vbc we will store our Terah home state right so uh it is now provisioning the infrastructure regarding the infrastructure it is uh the network which we are looking and in that particular Network we are looking to deploy our ECS cluster right so guys uh this is the official BBC uh module um publicly available uh at the tform registry right and we are going to use this particular side you can use any side of your choice but this in this demo I'm going to cover three AGS along with the private public subnets we are going to use one net gate we right so single net gate with true uh to save the cost anyway uh create database subnets enable DNS host and DNS supports this is our Network guys so pretty much basic stuff so once the deployment has been completed what it does I will show you the backend uh the happening at the back end uh stage uh you should be able to see uh some of the stuff available right till then uh what I'm going to do uh I'm going to navigate back to this area application load balancer regarding the application load balancer uh what we need to make sure um uh we should at https we are going to create the ACM certificate so guys uh this is the domain we are going to use right and um uh let me show you this part okay so let me open this domain so this is the domain guys san.com right this is the domain we are going to use right and we are going to uh set up the certificate acms through uh this command and we are going to validate by entering the DNS entries by updating the CNM records in our host public hostal zone right so this is my public hosted Zone guys uh I need to add the entries here right so let me show you first of all I need to run the particular command so guys our VPC is ready first of all navig to the VPC section simply refresh it you will see the terraform directory is created in that we have the VPC directory and our state file state file is stored here right so now what I'm going to do just to uh give you more understanding I'm going to um I'm going to take one more shell although we can also do the CD but I'm going to show you the stuff regarding um the different directories so instead of doing that so what we will be doing now CD into terraform ECS and we are now at the application load balancer section so we will be running these commands uh regarding the request so guys this is the certificate Arn what I'm going to do I'm going to copy this guy although this certificate is not yet valid but what I need to do let me navigate to this section and open it open the elb section main TF what you need to do is to Simply update the certificate and definitely you need to um update your domain or public domain if you have any so and if you don't want HPS you can skip this part as well so I'm covering some of the production ready stuff guys so we updated the um our um I mean certificate Arn and if I uh open this in another Tab and I'm going to uh navigate to the ACM simply type ACM here so navigate to the certificate manager right in this section you will see you need to navigate to the list certificates click the list certificate section we need to update the DNS record so we are actually requesting for this wild card. san.com and the and this naked domain name san.com right so this is the root domain so what I'm going to do these are uh same CNM record so it will provide as the certificate although you can add certificates for the multiple domain domains here right so anyway I'm going to navigate to my um public hosted Zone click the edit this is a cloud uh free uh account right you can create that as well if you want so it has lot of benefits regarding the management of your DNS so I'm going to save it let's do that and definitely it will take uh some time regarding the propagation and we will shortly see that um certificate is going to be issued which is the Wild Card certificate right so anyway uh so now what we are going to do uh I'm going to cover some of the stuff regarding uh press control C here right so let me do let me do L here so do I'm going to use the same command uh same command terraform init and terraform terraform apply D auto- approve right right so what it does it will do exactly the same and download the module in terraform directory so if iate to this section you see this is the directory where it is going to download the module from the report uh from the remote registry right and regarding that uh back end let me show you the stuff regarding a little bit more details so guys this is the back end uh back end we are going to use the same bucket right in that uh bucket we are not going to use the same directory you see terraform is the same but we are going to store our State Fire regarding the application load balancer in this section right so it will create a new directory ALB and store the state of the application load balancer in that particular particular uh directory right so uh we slowly building our infrastructure so if I navigate to the ALB section in the main TF guys uh what we are doing here we need uh the details of bpc so that uh detail we are going to pull from the remote state which is actually the VPC so how it comes it will navigate to that particular bucket in that directory you see I've already uh told you guys regarding the VPC state is store that it's going to pull the data regarding the VPC uh but if I naig get to the bpc section uh you can navigate to open this VPC in another tab in BBC uh we will get all output of our module and uh from that output it's going which is going to be the input for this module right which is the application load balancer which we are covering here so guys this is also the remote module uh I mean managed module which we are going to use and we are going to call this Cloud gigs Al we are going to use this uh Security Group regarding our HTTP and https right and excuse me so uh we do have listener rules uh one listener rule for Port 80 and two list rules for 443 so definitely this Ern we just already updated so if iate to this section the provisioning of application load balancer has begun and um um regarding the first rule on P 80 uh we are going to do the read direction right so um this is the redirection right um it is going to use the default um Target group so we are going to create the two target groups one is going to be the defaults I'm going to show you that below and other is going to be the uh for our application which is the python application so this is the name of the target group python application and one is going to be the default right so regarding the 443 rule guys um we are going to to use this section if the path is I mean uh path pattern is Slash and the host head is path sorry python d.com it is going to route to this so where these two guys are living these are living here so these are the target groups you see uh default instance you can call this any think you can even call this um default default Target group the name of this target group is going to be default and this is the python app which I've shown you Above This is the uh uh python app uh this one this one target group is actually we creating here and the name of that Target group is going to be python app to make it work with parget and the target type should be IP right and we we have enabled the cross load uh load balancing and this is the Bic stuff regarding that U delete protection I set it to false definitely you can enable that so if I navigate to this section our load balancer uh should be ready if I navigate to the U S3 bucket backend click the terraform you see One Directory has been created in that particular directory we have the application load balancer state has been stored right so if I navigate to ec2 section click this ec2 you will see one application load balancer has been available so this is our load balancer guys right so click this and you see this load balancer is now active and if you click this load balancer you will see two listeners 0 and 443 uh 0 has this rule redirect to https one rule along with that um we have two rules so click the two rules which we have already discussed so by by default uh I mean default rule I set it up with no priority which means this is the last it will get last uh by default and the priority for the first rule is going to be if the path is Slash and the H header is python. l.com it should push to this target group so navigate to this uh Target group let me scroll it down so these are the target groups right and you will see these s groups are going to be associated with our application load balancer which we have discussed right so by default the T group default has Type instance and this is what we need for our uh ECS application right so this is going to be python front end application right and uh okay this part is done now what we're going to do we are going to set up our um ECS cluster so I'm navigate back to the terraform code and we are going to set up our ECS cluster so I'm going to take one more shell okay guys to give you more understanding and uh visibility so I'm going to exact and take interactive shell into my sandbox um container right and I'm going to do CD into terraform ECS and we are going to set up our um ECS cluster so we need to definitely we need to have one cluster so similar stuff regarding that we are going to do uh which we have already done for our uh setup so what I'm going to do um let me navigate to the readme MD right so um and then okay first provision the e cluster right so let me do the similar command terraform in it and terraform apply Dash Auto Das approve right so make sure you should not have any typos right so uh let's discuss this part so this is our back end guys so our back end is going to be ECS you can add anything you want I've added ECS you can add ECS cluster as ECS cluster as well regarding this particular directory inside the terraform so um that is where it's going to store the state right and uh our e cluster state is going to be stored there right so we don't need this section so let me get rid of this and if I navigate to this terraform uh you will see the a new directory is going to be shortly available regarding our uh is cluster right so okay this part is done so uh this is also guys the remote module uh I mean you can easily set up the is cluster with the basic uh terraform uh TF file but I'm using the remote module so guys here what I'm doing what we need regarding this module we need the details regarding the VPC we need the details regarding the application load balancer because of the target group right so this is the manage module I've added the link of that particular registry and we are using um the latest version uh at the time of this recording so the name of the cluster is going to be ECS Cloud gigs so regarding the cluster setting guys I've um enabled the container inside you will see all the you know uh monitoring regarding your containers CPU the memory utiliz ization and the configuration regarding the log group we are going to store here right so forget capacity providers and farget spot you can uh read about these what these are I'm not going to cover in this video because the length is going to be too much uh in past I've already covered that uh the capacity provided SL so uh and the tags we are going to use the devops development right so our ECS cluster has been provision so what we are going to do now open one more Tab and let's try to refresh it first you should be able to see that ECS cluster is uh available and the state of that particular cluster is stored here if I navigate to um sorry I should type um ECS ECS simply type ECS here elastic container service right and then we are going to navigate to uh ECS so guys this is our U ECS cluster if I go to this section e cluster our e cluster is available currently we don't have any name space What name space it is using we are going to set it up and uh regarding that this cloud map is going to create the private hosted Zone and let me show you that part as well so we are going this is used for service Discovery and also we going to add one of the stuff regarding the uh service connect as well so uh guys uh now it's time to create our services right and uh now for that what we need to do first of all I'm going to uh describe my VPC so this is the VPC I've set up uh I need a VPC ID so like let me take one more shell and we are going to run those uh particular command into that particular shell right so into sandbox right and if I do MN into terraform so we have this directory ECS service which is the last part we are covering and some of the debugging stuff I'm going to show you so guys uh this is our bpc you will see the two um I mean this is the the default and this is what we have set up uh regarding the cloud geks VPC right in this particular VPC we are going in fact we have already set up our e cluster and we need a bpc ID regarding that so let me let me just paste this particular ID here and uh what I'm going to do I'm going to update this so we are going to create cloud map through CLI although you can create through terraform as well uh but we are going to create through AWC like there are different methods and different ways to set it up so I'm going to update the bpc ID right so let me do that we need to Simply copy this PPC ID and paste it here and run this particular command so what it does GU guys it is going to provision um the Cloud Web it will take some time and if you straight with list the cloud map right now you will see nothing so if I do uh do the list like um service Discovery list name spaces this is the command to list the name spaces okay for that you see currently we don't have any name space we need to wait for a while so let me show you from the section of the console okay first of all navigate to the Route 53 what happened at the back end regarding that command you will see the public and private host on living in this um broud 53 right and it will take some time you see this hosted Zone which is the private hosted Zone cloud map for service Discovery has been created with this particular command and and these two entries by default it has right so if I navigate to the ECS section you see the name spaces this is is the Nam space ID and if I run that particular command again so okay sorry uh run this particular command you should be able to see that uh name space is available which is the private hosted Zone uh these are the I mean IDs and Ern of that particular uh name space right so this is what happened so this DNS name is going to be used for the service Discovery right right so guys now uh what we are going to do or what we need to do regarding that NV get uh back to the readme MD section right so also uh what I'm going to do I'm going to create some of the secret which we are going to use in my container although these are not required these are just the sample stuff which we can use and just to give you the demonstration that the secret string with the AWS SSM parameter store we are going to use along with the basic normal normal naked environment variables right so if I have a secret and I want to make that secret available in the in my containers so regarding that what I'm going to do I'm going to create U command so I'm going to run this particular Command right so it will create uh a secret string so you can add multiple entries in that particular string but uh I'm going to create one more entry regarding that and let me run this particular man again so we need to update some of the environment variabl so let me show you from the UI so okay and if I click this guy if I navigate to the SSM simply navigate to SSM type SSM here right system manager in system manager you can store your secrets although you can use the a secret manager as well but for ECS I do recommend uh to use um s parameter store because it has some issues if you are use the programmatic um tform code or uh uh regarding the SEC right so this is the um ASM manager yeah should you should be able to see the parameter store you need to click this guy and the parameter store you will see um the two Secrets uh are going to be created and if I click this guy and the secret string is secure Ty is secure so if I click this guy you will see you should be able to retrieve your secret so what we need to do we need to Simply upgrate the Arn of that particular secret so if I navigate back to um my cluster and sorry if I never get back to my EC service section we need to provision uh the services so guys python app has a dependency of radis if RIS is not available this service is not going to work so what we need to do iig get to this section so let me show you the stuff uh like regarding the uh same stuff regarding of services each services will be living in our backend S3 as a separate directory and it is the state file is going to be stored separately so your infrastructure will be very safe right so and versioning is enable as well so guys regarding the redus I'm going to create this particular directory and the state of that particular particular radus is going to be stored here right so then uh this is the container def uh definition. DPL you don't have to do anything regarding that I'm I'm going to show you this is the uh default setting regarding log Gres environment secret P mappings I've already done that so what we need to do we need to update the locals first of all the same stuff we need uh the VPC details which we are going to touch from our remote State we need the details regarding the application load balancer uh but in Rus we are not going to make this service uh available available in the Target group but this um I've added here right although this part is not is going to be not played below but in Python app it is going to be available so uh if you don't want to add or remove you can do that as well so this is the um remote State uh I done typo here it should be it should be the eest cluster right so let me update this part so guys this part is updated regarding the remote state right and the labels uh sorry and then the locals local are the variables which we are going to use so the cluster name is going to be fetched through the remote State and we are going to you know patch the states and we're going to store this in this particular cluster name variable later on we are going to use it definitely below region is going to be us one you can update according to your so guys we need to um update the name space uh Arn Nam space name is going to be this one and you can use any name name space ID so uh you can simply uh see from that uh UI as well but I've added the command as well for instance um list name space command gives us all the stuff which we need so if I simply um type this particular command this is the information we need to update although this is already available in the UI let me show you that part so uh if I navigate to okay yeah Nam space this is the name space ID and regarding the AR and the name of that name space right so I'm I'm going to do uh the update so let me close this part and I need to update the uh Services section um first of all we are covering the dependency so red is um we need to update and then I'm going to show you the roles which are pretty much the basic regarding the permission to pull the images from ECS and SSM store definitely we need to add that permission which have already done that right so guys um this is the stuff I'm up dating here so simply I'm going to uh update this stuff uh sorry I need to add the Ern so uh regarding the ID let me do that so I've copied the id id right so regarding the Arn I'm going to Simply copy this guy right and I'm going to update this section regarding the ER so name is pretty much the same name which we don't want to update so the image is going to be redish and the container is going to be in the container name is going to R service name is going to be the same name I'm going to use so I've added this variable Cod is going to be 6379 so Guys these are the naked uh variables although these variables are not required I've added just to give you understanding right so uh these variables are required in Python appliation because that python application needs to contact uh this redis because it has a dependency so this is the second variables and you see the account ID is uh not updated so and the AR is pretty much the same so if iate to the parameter uh store here right so you see uh 7726 at the end so I need to upgrade this guy so uh simply I'm going to replace contr F and simply I'm going to replace this section both right and let me also upgrate this as well it should replace both but it has not done anyway so I need to update this section because we have set up the two parameters if we don't update this will gives us an error so just uh this is something which uh I'm showing you to give you the understanding how you can use the secrets right and um the basic environment variables and regarding the cicd um I do not recommend to use this terraform code definitely regarding CS they already covered the videos you can watch those particular videos right so guys uh this part is is updated and um some of the stuff which we need to use regarding the service uh Discovery section so uh we need to register the service so if you navigate to this readme MD so what I'm going to do I'm going to uh list the services uh service Discovery Services currently we don't have any services so let me um uh list the services so currently we don't have Services what I mean by that uh instead of creating that uh particular um Services I'm going to create through aw CLI so um let me show you this command so guys I'm going to create one service the name of the service is going to be redis and this is going to be available in that particular name space I need to update the name space ID right and um we need need to update this uh namespace ID configuration as well so what we need to do simply copy this guy right and and if I P this command here so so what I'm going to do guys I need to update this area regarding the name space ID so previously we done through the from the CLI and if I not get to um this namespace area right Nam space ID Okay click this guy so this is the name space ID you can simply copy this uh and I'm going to do the right click here and then also I need to update this names space ID so this is the names space ID and the name of the service uh which is going to be uh available in our uh name space so let's try to run this particular command so guys uh this is the uh service ID and this is the Arn we are going to copy this Ern right and we are going to update this particular AR in our code so uh if I scroll down you see this is the task definition we don't do we don't know we don't need to do anything regarding that we already uh set up all uh through the you know from the environment variables so um and uh this is the cloudwatch log configuration which we are going to set it up so uh regarding in uh this service registry I need to copy I need to register that particular service right with our ECS uh service so I'm going to Simply update the AR of that particular service so let me show you so make sure this is correct right and so guys this is service this service is going to be registered if I get to the name space area so click the name space section right and you will you will not you might not able to find the services but if you do run the list command it will show sometime this UI uh not show the services available but uh I've added uh the link regarding that travel shooting part U the stack flow as well so uh if I run this particular command you see which I've already done it so we can simply able to see this services are available so redish is the service you see the name of service is this which is now available anyway uh now we're going to set up the also add the a service connect configuration you just need to add enable through local. namespace Arn this namespace Arn is going to be the one which we already set it up above and Discovery name is going to be the container name right and the port name is going to be the one which we already added in our um container definition. TPL so uh let me let me show you part this is the this is the stuff which we are talking about this is the port name right so okay this part is pretty pretty much done and we don't have to do anything and the DNS name is going to be the container name and the Nam space name which is the Sean M.C and the container name is going to be R which be set up and the p is going to be uh 6379 okay uh regarding the network configuration so uh as I told you guys we don't need a load balancer so this area is not available here so I commented out to give you more understanding and visibility uh and this H check is not going to be applicable here so in Python application we are going to add that in our Target group so that section is going to be available here so these are the life cycle policies anyway I've added the service auto scaling as well if you want to set up definitely the auto scaling and the scale in scale out we are going to set it up on the basis of CPU and the memory utilization right and also in my python I'm going to show you I did one more thing regarding the number of requests on the application load balancer which we are going to cover so things seems to be okay in case of any error we will debug it so let's try to create our service which is going to be the redus so let's do the similar stuff terraform init and terraform apply Das Auto Dash approve so guys uh this will create first of all a task definition and that particular task definition it is going to add container and then it is going to create service also it is going to add I mean connect with service so there is a typo let me update this uh it is going to register the service uh with our um name space which is which we set up in our um cloud map and then we are going to use the AWS service connect as well so let's navigate to the UI uh let's navigate to okay uh this is our condition regarding our back end so if I navigate to this terraform you see currently we have this setup so uh regarding the task definition you see shortly U if everything goes well uh one task definition with the name redis is going to be available right and okay uh I forget to show you guys the roles so uh regarding the roles I set up the pretty much the basic stuff uh is the Eis roles. EF so this is a role which is going to assume uh by the ECS U container right and uh I mean t execution rle it has the policies regarding the authentication token to um create log groups um has the ability to get both the secrets read on secret from the parameter storage we have set up added additional stuff regarding secret manager and also get image from the ECR as well if you set it up and then we're going to do the attachment right so this is the stuff so anyway uh it is creating that part regarding our um redis uh let's see and uh okay I'm going to check it up so if I navigate back to the task definition section you see task definition is now available so uh in this particular t uh task definition we have one redish and we have provided the image and other settings we can see the Jon as well regarding that right the port mappings which we have already defined the other stuff regarding the envirment variables and and the parameter store so we can see from that container as well so if I click this container and scroll it down you see these are the environment variables and uh some of the stuff which is secret is coming from the secret parameter the store right so uh it is creating seems to be an error okay so usually it will it will show you this error that service already exists so regarding that I've already added um uh added the code so if can navigate back to uh now to names space section so that refresh you see all those service is not shown U but it says Service already exists so we have already added the error so uh what we are going to do uh let me debug this part and I will be back with you guys shortly so guys uh this is the error which usually come uh with the code uh regarding the terror form so um what we are going to do first of all we are going to list service and we are going to uh uh I mean get R of this R service and try to recreate that for service right so so let me do that so what I'm going to do here first of all I'm going to list this particular service which we have set it up and then we are going to uh get the ID of that particular service right so guys this is the ID right and so this is the ID right so what I'm going to do um I'm going to get rid of this particular um ID which is the service which we have already registered press space and we are going to simply delete this service right if I do list the services again so I should not be able to see the services registered so I'm going to use the same command and I'm going to Simply copy this uh particular Command right and I'm going to I need to update um regarding the Nam space ID so I'm going to use this particular Command right and if I navigate back to this section so if I navigate to the names space and um this is the name space ID simply copy this guy and simply paste it here and also I need to update one more time in this area as well so let's do that just wait for a while so it it has been created so I need to Simply uh copy this AR just to make sure so if I navigate to this section and right let right to press control and control V you see uh we have this same a right so let's try to run the par command again right we have already initialized it so I'm going to get rid of this part initialization to terraform apply and auto let's see what's happened regarding that particular error so it do it does come even if you create through the terraform so I've done some debugging regarding that but uh this is actually um coming on regular basis so let's try to get rid of this I've added the SL uh stack of low link regarding that so let's see uh What's happen again so it is saying creating so if it work quickly it should be just able to create the service quickly otherwise if it is in uh this state it gives uh the same error again and again you see uh it's uh it blown up to 20 second 30 second so it it will spit uh that error as well uh right so uh let me show you this part again because you see the time has eled too much and it is going to create that and again because if you see the issues that um if I get to this section uh regarding the cluster our service is not yet created cre it and it will gives an error regarding that so so let's uh let's uh debug this guy and I will be back with you guys shortly and you will see definitely this sort of error regarding that although if you see the terraform code it is actually saying that we going to set up all the stuff regarding the is service right and uh um then the resources section so I'm going to debug that and I will update the issues so as I discussed guys this video is not not just relating to uh H Serv Discovery I'm going to show you some of the debu regarding the terraform along with the service Discovery in ECS as well so guys I have uh fixed that regarding the fix I need to show you what I've added I've added a depends on on the auto scaling so it has now dependencies on the EAS service but the main part is now I need to show you uh what I have done I have deleted service and we have get the Arn of that service right so um you see uh let me show you this part if we list the services so uh this is how we list the service right so if we already create this service what it does it will throw an error that service um has been already added so from here what would be the Arn we need to Simply copy this Arn right and uh and we need to update this uh in this section where we register our service because what it does it try to create that so we already done that and then what we are going to do we are going to simply delete it right so that is where we delete the service so let me uh rerun the uh rerun the code again so you see the service has been created successfully the law configuration environment variable and all this stuff so if I need get back to the section so this is the debug I need to tell you that uh create service take the air and put the air in and simply delete the service this is the solution right you can note it down so uh regarding that I have already added commands and I've already updated the dependencies uh which we need to have so if I navigate back to the cluster section and um try to refresh uh you see one service has been already added and you see we don't have any error and we have don't have any no changes in our uh instructor report so guys this is the de but it has one issue what the issue is that I need to tell you that we are using SSM parameter store and we are using the default DPC Security Group and this radish is going to be continuously created and destroyed and this task is U you see although it is in pending state but shortly you will see some of the stuff in the stops uh area why it is that U because it is unable to pull the um uh Secrets right so uh you will see uh why it's happening although we have already added the right uh policy section right and um but what I need to tell you that regarding the task so um this task if I navigate to this task section and if I click this task and if I navigate to the network binding right and let me do that so guys uh if we navigate to the configuration area and if we click the networking section right and here if you see the default Security Group why it's failing uh I'm going to show you the stuff because SSM parameter is not living in a youc it is a remote service living outside if you see the inbound rules and outbound rules we don't have anything so if you have a good understanding regarding the network how the SEC group behaves you can easily do that so the issue here is guys we need to update the outbound rules because the outon traffic to um SSM is not uh happening so what I'm going to do let's say um type A for all traffic to every if I do that that the provisioning of R is going to be okay because currently what's happening we have uh added the SSM parameter store and is looking to get fetch the details in Security Group we don't have anything regarding the outbound uh traffic so it is actually unable to talk to that particular service so it is throwing up an error so uh this is some of the stuff uh you lots of people do struggle uh when they set up SSM and uh secret you know in their container section so if I navigate to the name spaces and if I click this guy um you see red is although it is active but uh you see it is in progress and if I navigate to uh this section and you will see some of the traffic help and all the stuff uh is going to be available but if I navigate to the cluster section click this uh area red is you will see uh I mean uh tasks and uh we have to wait for a while because now we added something which which needs to be fixed so if I click this stops tasks you'll see two stop um task out there but what uh I mean EC is going to do is going to keep restart um those field stuff so unless it is become healthy right so currently it is in pending state so to see our ECS service connect is also in pending state so let's try to refresh wait for a while so guys I've intentionally added this stuff although with terraform uh I mean I can simply um um add that particular Security Group outbound traffic uh this is uh as I discussed uh to give you more understanding regarding the debugging area what we need to do and how to fix those errors uh to save the time as well so again let me navigate to the Rish section withit for a while do right just sping you need to wait for a while or might be uh I'm going to pause the video see again one task has been provision it is provisioning and let's see if we get any error regarding that so it is in pending State and let's see I'm going to go guys I'm going to pause the video and once this task is healthy I will be back with you guys and sorry fun thing for I forgot to add um if you're using the private host Zone sorry if you use the private subnets uh you need to configure the net Gateway uh and the is going to route through that n gateway to the SSM service you need to update the security groups and if you are using the public subnet you definitely you are uh you have already set up the igw which is the internet gateway and you need to Simply update the security groups regarding the configuration you see uh the difference is that we have now our services healthy Rish and ECS service Discovery so uh this is something regard regarding the redish and uh if I navigate back to this section regarding um the redish so you see it has completed successfully which means the health of redish is in a good shape we are good to go so uh you see one replica is there and it's completed and the set is now green so this is something you need to have in your mind regarding the setup now what we are going to do we are going to to our last part regarding the deployment of the Python application right so uh let's do that so guys if I navigate to the S3 bucket I need to show you you simply refresh this part you see we have four objects so simply scroll it down this is our ECS um service Rus application the state of this is stored safely here with the versioning enabled right now what we are going to do um this time I'm going to do CD instead of opening one more tab so what I'm going to do I'm going to navigate to our python application and we need to do the updates similar updates so now what I'm going to do regarding the updates for instance uh uh this area I need to update uh for instance I need to update this section regarding the uh namespace Ern and uh I mean namespace is to be same but let me click this guy and I'm going to simp close this section regarding Rus now open the python and if I get to this python section you're watching this directory because uh previously I've already tested this so that is why so I need to update or get rid of this existing stuff because the Nam space has uh ID is updated and namespace Arn and the stuff regarding the this area regarding the SSM parameter store you see account is different now I'm going to Simply update the account here so click this section and I'm going to click this account area right and I'm going to Simply to update here press contrl F and replace um this area right so let's do that replace all so this part is is done and this is really important this is this python service is going to contact the redish it must be available and should be discoverable right so we don't need the service Discovery regarding the python because it is the uh front uh facing application which is going to be publicly available let's try to run that and um this is going to be registered in our uh Target group but regarding this registration I also added one more issue and I'm going to show you that deeper part as well so terraform init and terraform terraform apply Das auto- approve so uh this will create this python uh task definition along with that it is going to add the container and and our environment variables and secrets along with that it is going to create one service but that prod service is not going to be healthy and I'm going to show you that why it's not getting healthy and this is also the issue lots of people face regarding the eest setup although they all even setup manually or through through terraform um what they need to do I'm going to show you that part so so uh This Is The Stuff guys we are setting up and uh let's see so it is going to be the cloud watch log group and Service uh if everything goes well it is going to quickly create that particular task definition service and CPU the load balancing and everything is going to be quickly done right so it has done that already so if I navigate to this section and uh simply refresh it right you see on our Python and redish uh directories are available regarding our backend if I navigate to the Target group section if I click this guy shortly it is going to register service here but it will have some issues and I'm going to show you why um okay let's navigate to the cluster section click clusters now we have two applications services available but this python is going to be unhealthy it is all in progress and some of the stop T why uh you will be see the stop task I intentionally added the issue here to show you guys first of all let me show you uh because it it is going to register this guy here in the TD Group which is the front ping app so regarding one and three I need to tell you uh if I navigate to this load balances section click the DNS name and if I navigate to this section I need to update this python Das app and I'm going to Simply update the DNS uh uh regarding the application Lo balancer although I already tested in past but this is the sandbox environment things are changed so this part is done so uh if I navigate to uh load Balan section right everything seems to be okay the rules which have already covered and now uh I need to uh show you guys regarding the stuff right so let me uh close the stuff which we don't need to give you uh more understanding the easy understanding so okay now let's try to just refresh this is the hosted Zone it has created so let me see uh and okay this is the security group and uh regarding service this is the python task and some of the stuff we need need to see regarding you might see that this is healthy but service is healthy it is actually conted at that name space but if we shortly see some of the stuff regarding um the state which uh the load balancer is going to do so it is running which means it is going to do the registration in the load balancer whenever we get to the Target groups python it will try to do register itself in this particular uh area you see it is unhealthy why it is unhealthy they need to fix that so um first of all um if I navigate to this task it is healthy so um if you scroll it down so what I've done I've intentionally added the public I guys in the production environment you need to make sure you set up the public IP set to false along with that you should be able to add I mean you should add I've added intentionally the public posted I mean public subnets you should use the private subnets right this is just for debugging purpose and what I'm going to do um I'm going to navigate to my security group inbound rules and I'm going to um add one more rule regarding the TCP right uh TCP Port 5,000 my IP click this guy paste the IP of this this particular task just for debugging purpose okay um might be that part task is removed because tet group to remove those so let's try to refresh it okay now it is still there so um we have this uh connectivity issue um in okay connection reset by uh Pier so might be this is not available okay okay because uh it is going to delete the task so if I navigate to the services section Okay click this guy and take the IP address of this paste it here and 5,000 wait for a while and also okay connection reset so guys uh if iate to this uh Security Group what I'm going to do I'm going to navigate to um add one rules regarding the TCP 5000 I'm going to allow the AL B simply type ALB here right this is the application load balancer save this guy so let's see the status and um what's happened okay and uh wait for a while so new um it is actually removing the previous uh stuff so it is now initializing and the health check is the port which we already set up on the port so wait for a while okay it's still unhealthy which seems that uh we should be able to see our uh redish and uh it is seems to be not working I need to debug that part as well why it's not working so guys issue is regarding the application so if you see I navigate to this uh service radis right and in this radish service you see it has successfully registered here so this is the service uh Radice and it is registered this is the cluster name this is the ECS s name so it is successfully registered so issue is regarding uh the exception uh which is coming from the application side so uh we need to fix it and um regarding the networking part you see uh you'll get this error once this is fixed uh at the application Level uh we have no issues so regarding the uh Security Group uh we have already added uh stuff for instance U allow 5,000 Port uh from the application load balancer but but the issue is at application Level so that's why it is not working right and uh I need to check that and it it will take some time but this is the basic networking uh for instance if you navigate to the security group even I've allowed uh all traffic and let me just do one more stuff uh is to allow all traffic from anywhere but the point I just want to make here is to that the issue is uh regarding that application Level let let said we have actually allowed all the traffic from anywhere and um the last thing we are going to do uh we are going to check so it is draining the previous uh stuff so um I mean so this is currently uh available so if I navigate to um the cluster section and if I click clusters and where we have this python application right so uh if when to the task the task it will allow the uh it will allow um it um this task uh public IP so let's try to do that but uh definitely need to check at the application Level so this application has uh definitely some issues but uh let's try to debug this uh how we can uh do the fix right so guys I've debug it and uh issue is actually coming from the application Level the reason I'm saying that uh if you navigate to the task definition uh section of um the application python application right and in the python application what I've done I've updated to uh debug uh this part I updated the task definitions manually and you see these are the task definitions so previously we have this this one and this two and if I use this task definition and in this particular TK uh definition uh if I click this guy so what I've done I updated the DNS name to R source to R source so it will say the DNS resolution will not work so we can check that so let's try to Simply update this and what we are going to do uh we are going to navigate to the Clusters and click the cluster click um this guy click python app and simply we're going to do uh the update and the update is going to be with the task uh definition revision two and and you will see previously the error was the exception so which means uh now uh if uh the new task so if I navigate to this task remove the um uh I mean uh stop this existing task and stop the selected one so what it does it is going to provision new task shortly and then we are going to test that particular IP address and if we see uh we should not be a ble to resolve it so regarding the resolution we don't have any errors so uh this because I'm going to prove that so um this is exception coming from the redis and uh um because of the some of the connection but previously the uh redis was uh actually it is actually the connection error but we have some errors regarding uh the redish you know um so if I test this part uh regarding that so uh if I navigate back to this section so uh okay services and python I just try to refresh it and uh you will see shortly one Tas is going to be uh provision and what happen is going to do the registration from here in the Target group so it has now deregistered so you will see shortly one task is going to be available and we have to be patient with this um as I mean Services updated and shortly it is going to uh provision one task so let's try to refresh refresh so deployment section you will see shortly uh one deployment is going to be available you see previously we have these fail tasks and it is in the draining stage uh this why we are unable to see one task so it is going to provision that with the new U you see um the new revision so um if uh we navigate to the networking section uh and just try to refresh it it will assign this guy public IP so if I do simply paste it here and simply update the IP to sorry code to 5,000 you will see the DNS resolution is uh not going to work so let's check that okay seems like my IP is updated so I need need to Simply update the inbound rules and check my uh IP address regarding the port 5,000 so K seems to be okay good so we have to wait for while to provision this you see now you see uh name does not resolve so uh previously the error was different which is this one so which means uh we have R exception uh right so uh regarding that so anyway if you like the video like And subscribe the channel have a nice day thank you

Channel: DevOps Gang

Views: 282

Keywords: Terraform, ECS Service Discovery, ECS Service Connect, Cloud Infrastructure, Deployment, AWS ECS, Terraform Tutorial, Cloud Services, Infrastructure as Code, DevOps, AWS Tutorial, Cloud Computing, Terraform ECS, Service Discovery, Service Connect

Id: mFXEKI6oumE

Length: 77min 34sec (4654 seconds)

Published: Sun Dec 17 2023