Kubernetes | Terraform | AWS EKS Cluster Setup Using Terraform Cloud and GitHub Actions | GitHub

a hi in this video I'm gonna show you how to deploy AWS eks cluster using terraform cloud and GitHub actions as you see in this PPT this PPT depicts My Demo here I'm gonna show you like how you can you know write the terraform code and set up the you know the CI CD pipeline at GitHub to deploy the AWS you know eks cluster management with using infrastructure code right so here this PPT design diagram which will show you like you know how I'm gonna set up the infrastructure at uh you know at the GitHub and terraform Cloud to manage the eks cluster right so the solution would work something like this we use GitHub for source code storage purpose that is our infrastructure as a code terraform code or terraform file storage purpose yeah and also we're going to Define our GitHub actions uh you know definitions in the in the GitHub itself right so that's where you know the developer Parts come in pictures where the developers will be developing the you know the uh the AWS eks cluster management with using terraform yeah so here we're going to cover value very basic like how to spin up the AWS infrastructure that is AWS eks cluster infrastructure with using terraform and eventually makes it like you know infrastructure as a code managed right and then you know so then we will have a GitHub action workflow file which actually compiles these and runs these data from file and eventually you know So eventually it also interacts with the terraform code so we will have a terraform Cloud subscription where we will create a workspace uh in that workspace you know we're going to store the state of the AWS eks cluster right and that's where the the even the CSD pipeline will interact with the terraform cloud and the terraform actual the you know invocation or the run of the terraform Cloud happens at the uh you know the run of the terraform files happens in the terraform Cloud which actually spins up the you know AWS eks cluster right all right so that's where the flow is something like this you know we will have a source code here in the GitHub we will have a state file in the data from cloud and we eventually we will see the Amazon eqs cluster up and running file yeah so that's the basic I know the Oreo about the you know the solution now let me take you to the my source code okay so this is my source code which I have already cloned in my you know so in this Visual Studio code the my you know GitHub action or GitHub repository is something like AWS eks Labs I'm gonna share these files in my videos description you can find it uh these source code files from you know from the uh from the video's description you know links being given down you know given here yeah so here if I go to the my repository which we created for this demo purpose that is AWS eks Labs okay this is a public repository where it it actually can you know construct the you know this repository has the files and folders needed to deploy the AWS eks cluster right so the dot github.workflows contains the you know the the workflow files which actually compiles the terraform code and then the models is nothing but not the models which is needed to deploy the eks cluster Main and output and provider for you know files are are the files respectively which actually creates the eks cluster right so I have cloned this repository in my visual studio code so this is how it looks now what I do is I'm going to walk you through the code you know very basically and to help you to understand like how does the invocation happens within each other in the terraform file and with the you know the Gate of action file and then we're gonna show you like you know how to create and set up the you know the TFC workspace as well finally we're going to run the you know the uh these workflow file and eventually we see that you know the infrastructure is up and running fine in the in the AWS account so as I said you know so the first one let me walk you through the another source code first that is the piece of code which is responsible for managing the infrastructure right so for now we ignore dot GitHub for slash workflow meantime yeah and then we're gonna go to the other files and folders that is modules and then main.tf file so the main.tf file here in my code you know it actually you know calls the model in this is the model invocation happening in this main.tf file so here we have created the uh I know the two models one is the um basically eks uh the other one is the VPC as the name says eks this is the code which actually creates the eqs cluster right and as you see you know I'm invoking the modules because I have created a models like models right and it that's the reason the source path is is pointing to modules.aks then we have defined another the subnet required for eks cluster that is public subnet note that the cluster which I'm spinning in this demo is is public cluster while in my next iterations of my videos you know I'm gonna focus more on your private subnets you know private eks clusters actually as well yeah so here we will have the VPC ID cluster name you know the endpoint private endpoint yeah then we have the you know public uh cidr yeah and then the group node group name the scaling desired State Yeah so basically and then we have a maximum minimum scale set definitions and the instant stuff here I'm using T3 small because this is a demo yeah so and then we have VPC model so VPC model is pointing to the folder structure that is if I expand this one we have a eks and we have a VPC right and that's where it is pointing and it has these many parameters that is tags instance you know the the in in instance tenancy vpcc IDR um and the access IP address right and and the public cidr ranges so basically uh you know so this is the vpcc adder we are breaking that into two public subnets like for slash 2424 right so we are creating just a smaller chunk as the public cider out of Forza 16 and then we have our route table cider range as well so as I said you know this is a public cluster that's the reason we will gonna have certain further resources of VPC to make the eks cluster public all right so this is all about the main.tf file if I go to the output so we are actually outputting certain things that is cluster ID cluster endpoint and cluster of names yeah and in the provider block so the provider block is very very important block of of your you know the infrastructure as a code so here we have a terraform Block in the terraform block you know we are actually uh dumping you know we are actually accumulating the required providers so here the providers are AWS random and kubernetes right and then within that there is a one more parameter in the terraform called back end in a sense where actually you are storing the state of your you know the the eks cluster so here my uh state is you know storing getting stored in a remote place that is and the app.t terraform data which is nothing but you know it's a SAS service from from hashic cop which actually you know stores the stores the state of your infrastructure remotely uh in a space called workspace okay so generally the workspace yeah so we give the workspace name as AWS cluster I created this workspace and I have configured I'm going to show you immediately after this right and then we're going to have the data that is the cluster data cluster authentication which I'm just collecting to set up my kubernetes providers in my next iteration of the videos this video I'm gonna help you with like and how you can deploy the applications say Observatory applications are normal uh you know the the need you know the cloud native applications of kubernetes on eks cluster how you can deploy those okay so we I will show you ahead in my new videos okay in my further series of videos around eks so this is what it actually set up the provider R we can consider like you know this is the block which actually creates a cube config file for you right and then we have a provider for uh for AWS I'm using deployment in US West to region that is Oregon reason then we have a random provider resources okay all right so these are all the you know so basically these are three main uh you know three dot TF files which are created directly underneath the the source code repository this is our source code repository and underneath that we are directly having these three you know the files and then I'm gonna walk you through them the modules folders and file structures as well first module let's go to the models for slash eks model so as the name says eks model so this is these are this is the model which actually creates the resources needed to spin up the AKs cluster so first one is you know we are creating the AWS underscore eks underscore cluster that is that is basically this is the you know the terraform code which actually creates a cluster for us here we are providing the name and the role name so these are all the values coming from the from the main.tf file uh when it actually invokes the module right and the name role and we have a VPC config in the VPC config we have the subnet IDs endpoint Public Access public cider security groups blah blah and all and it also it depends on the you know the the role here and right so to create to attach a role you need to have a policies attached to that AR and you know the role which we are creating here and that's the reason it also depends on a certain policies so this is actually creates a eks cluster now let's go to down the another one so cluster creation is not enough so we also need to create a e case node groups as well so this is the block which actually creates AKs node groups it contains the cluster name you need to provide you need to provide the your node group name node rule Arn and the subnet ID instance type yeah and then the remote access configuration scaling configurations and debates on configurations have been given in this file okay so basically this is actually creates a the node group of your AKs cluster that's the reason we have a dependency here if you see here that is AWS cluster quick labs and then the names yeah that's basically that becomes a implement dependencies yeah all right so this is AKs cluster then we are creating the security group right so in the security group we are adding the Ingress and egress rule that is we are actually allowing the TCP Port that is 800 ports to to the internet we have egress which is actually allows everything that is we are creating a custom uh another node Group which will be attached to a node group's basic Custom Security Group which will be attached to our node groups so how it is attaching you can see here basically Security Group so if I select the security group let's see how it has been pointed so these security groups are being consumed down the line if I go down so we have a dependency yeah basically this Security Group is is consumed in the remote access configurations of the node groups all right so that's about our security group configuration then we have IM rule so we are actually creating a two IM rows one will be attached to the cluster one will be attached to a two two basically to the node groups one is the IM role which The Trusted to the AKs cluster and that will be attached to the eks cluster yeah and this um you know so basically this uh role will be attached with the policies like Amazon eks cluster policy plus Amazon eks VPC resource controller policies will be attached to it and then we have a you know so we also have a node group IM role which is trusted by ec2 machine because node groups is constructed by the ec2 machines only right and then we are attaching the three policies to it that is eks worker node policy cni policy and then the you know the registry the is to container registry read only policies have been attached to in this one yeah all right so with that we have a two other files which actually print something that is you know does the output and then we have a variables which we are you know basically given as input from the main.tf file all right so all these like public subnet vpcc added all these information would be fed from this main.tfl you see there are the these are the parameters which you are passing is equivalent to the parameters which we are declared in the variable.pfl all right and similarly let's go to the EK let's go to another folder or another module underneath models folder that is VPC module so in this model we are actually creating the VPC and subnets plus internet gateway because this is a public subnet so here we are creating a VPC with the cidr that has been given from the VPC model invocation yeah and then we have internet gateway so we are creating this internet gateway here so if I can do this like this yeah we are creating the internet gateway here and point in attaching that internet gateway to the VPC which we created above then we are you know actually finding out the availability zones of the region okay those all updated zones will be used down the line to spin out to submarena to actually distribute the subnets of the VPC that's the reason here so this is like a dynamic uh subnet creation resource block actually which creates a two subnets because we have a subnet count equal to true and it is attached to our those subnets have been carved out of VPC which we are creating above then we have a default route table so which is actually creates a route table in that one we are actually adding the routes that is uh you know this is the cider route and Gateway route yeah basically something is been configured here and then finally we are actually associating the the route table to the to the you know to the subnets so the AWS subnets if you go to the AWS subnets here and in that one we have a quick lab subnets and we are actually you know the attaching this uh route table routes to uh to the subnets all right so basically that's all about the configurations of the VPC in the sense this piece of code is actually helping you to create a uh you know the uh public eks cluster in the sense this is bare minimum configuration you need to be keep in place to make the AWS eks cluster up and running fine yeah all right so this is smooth enough that you know it actually creates a eks cluster while you you know while your requirement could be changing here in this pulse you just need to go ahead and create the respective resources make this model more usable or make this model adaptable to your use cases as well then let's go to the you know the github.workflow file which actually you know the file which actually invokes or compiles our terraform file and spin up the you know spins of the AWS eks cluster so here I have a three dot yaml file or basically workflow files don't get confused I am just mimicking the reusable workflow methodology to to to run those you know the the AKs cluster right so here then my name is zero one eks cluster and this eks cluster will be invoked on workflow dispatch which means that you know we'll be invoked during the you know if you are manually invoking then you have a plan and then you have apply as well so uh plan is pointing to the I have a two files with the name like AWS TF underscore plan and AWS TF underscore apply right so all those those two files will be invoked right basically plan will get executed first then we will you know execute apply so first let me go to the plan basically this actually does nothing but it actually you know just compiles these files in a sense it just runs the terraform plan from the folder that is AWS eks labs right so here the name is Bill TF plan and the workflow call so when on workflow call you know it actually expects you know it actually expects the two or three inputs that is tier version and GitHub environment yeah since it's a reusable I have created environmental option as well here all right similarly let's go to the the jobs and the jobs it's very straight for our job so it it runs on the Uber to let test and environment is is the environment which we are parameterizing then runs on a Bash yeah then we have a steps first one is check out setting the terraform we are setting the terraform with the TFC CLI token so this token I'm going to show you how you can create that and then we have a terraform in it error from apply commands yeah likewise uh terraform apply does the same thing but extra thing that it does is terraform apply yeah so that's also basically I have very quickly walked you through the another code base that you need to set up and how these are connected to each other to you know to spin up the AWS eks cluster infrastructure at AWS now let me take you to the my uh you know the my terraform Cloud subscriptions this is my terraform Cloud subscriptions I have an organization called Cloud Quick Labs which is recording to my channel name and then we have a workspace in that workspace I have created a workspace called AWS eks right so in this workspace if I go to the variables I have set the uh you know the as usual the variables that is AWS key and secret will be set here okay so don't worry so I have exposed these secrets anyways I'm gonna delete these secrets I mean after the my video has been created you don't need to try it all right okay so basically this is how I sent it my secrets yeah and my um so if you see here I have already compiled the code and the last compilation of the code has went Green in the sense it has successfully applied an infrastructure has already been Sprint up here so if you go to the update you know see the you know apply finish in the sense my infrastructure is up and running fine yeah since we made some changes I'm gonna show you that you know it this piece of code does work as expected and it also spins up the infrastructure in the AWS you know as as expected as well right so I will say like you know demo changes I will just say demo changes I will commit this and I will synchronize into the to the Upstream Branch right once it gets synchronized I'm going to invoke the workflow file manually so if I go to the my AWS sorry this is my you know the GitHub repository in that one awsck slab is my you know the repository which we were talking about and the folder structure that you see in the visual studio code is equivalent to whatever you see in this uh you know the GitHub right so you have a main output then provider then we have a model and workflow file okay so if you go to the GitHub actions right so here is the your workflow so this workflow is is you know you know the manually invokable right so um yeah so basically as I said you know so once the your workspace is being created and you configure the environmental variables right and what you do is you know you need to do one more thing is it's basically go to the go to the you know so go to the user settings and have a token so basically I have also created a token uh something like this which is uh going to expire in a 25th November all right and this token is been used and set as a secret in the in the GitHub environmentation this is my repository in this one I have created an environment called Dev in this environment you know I have created a secret called TFC API token so you need to you must do this or else you know the code invocation gets failed for your information right so with that note I have already covered every and each and pieces of the work that you need to set up to spin up the eks cluster all right all right so with that note what I do is I'm gonna go ahead and run the workflow to give you a real-time demo so here I just invoke the eqs cluster let's see how does it works if you see here the the invocation of the eks cluster uh basically the compilation or the you know the execution of the terraform code it has now started from the GitHub actions and the GitHub action is performing these stages so I have a two uh you know stages here that is one is we'll plan it and then another one will run immediately after it yeah so basically if you see here the currently we are actually showing seeing you know the terraform plan and and if I go back to the uh you know terraform minute it actually successfully connected to the remote backend that is TFC yeah then we have a validation is also fast and then we have a plan which is getting executed right so currently it says there is no changes right so with that note in a sense basically since I didn't modify in the terraform Cloud uh terraform uh so basically I didn't modify the terraform files so that's the reason the infrastructure matches and it identified that you know there are no changes we should expect same thing uh from the from the terraform apply as well so let's see how does the terraform apply works now so if you see here the first plan got executed then it is running the apply so why did I do in a two stages because you know the plan will just draft your infrastructure and it will tell you hey I'm gonna do these changes at that time you know you can see you can read the summary and then once you are satisfied with the summary what terraform is telling you then only you can click on a button like apply that's the reason I divide terraform plan and applying to two stages so that you get a gap of approval so in the sense you in between you can keep a approval options as well in a sense to apply you can keep it a approval option so that your infrastructure remains safe always and your application ups and running fine always yeah so here you go the the infrastructure is already been matching the you know the configuration all right now once the apply is up and funding fine I'm going to take you to the my AWS account and we're going to see the infrastructure so looks like terraform apply has been gone green now um which means that you know it has applied successfully so let's wait for a couple of seconds to complete the terraform apply and then I'm gonna take you to the my AWS account as you see here even terraform applies says no changes all right looks like the pipeline has gone green it is taking time I think yeah now it is done all right looks like now it is done if I go back to the here you go the plan and apply has went green yeah all right now let's go to the AWS account and see the cluster So currently this is my AWS account where I was producing the infrastructure that is e case infrastructure with using terraform you see the my cluster that is of version 1.27 is being a pen running fine if I open this you know it looks like every configuration we have created a eks role this is like this we have a resource we have a compute if you go to the compute we have a node group as well all right so and then the node groups everything has been as expected configurations if I go to the VPC we have a VPC with a name like Cloud Quick Labs right and it has a clash range of you know photos 16 10.0.0 and we have a two subnets yeah um name is something like this if you go to subnets we have a two subnets of of VPC this year yeah and then we have an internet gateway as well which we created and then we have a you know the route table as well all right so which actually what does this mean since I have so basically we have mayonnaise to create the you know the uh AWS eks cluster that is managed kubernetes cluster infrastructure is now managed with using infrastructure code with using you know GitHub GitHub action and terraform Cloud all right so with that note I have successfully shown you the things new to be shown in this video finally account request please do subscribe my channel that would really encourage me a lot so with that note thank you thanks a lot and see in the next video

Channel: Cloud Quick Labs

Views: 5,578

Keywords: #cloudquicklabs

Id: HeX_vNQv7CE

Length: 21min 57sec (1317 seconds)

Published: Sun Aug 27 2023