Real-time End-to-end DevOps project: Deploying an EKS Cluster with Terraform and Jenkins

Namaste everyone and welcome back to our channel  so today we are back with another terraform   project so basically we are going to deploy  an eks cluster on AWS using terraform as well   as genkins so instead of directly applying the  changes with the help of terraform we are going   to deploy the changes with the help of genkins  which is a cicd tool so if you're looking to   enhance your skills in devops and automation  then this channel is the right place for you   do make sure to go through our content okay and  if you found it useful do make sure to like share   and subscribe as well so now without wasting any  time let's get started now first of all before   actually beginning uh let me tell you why do we  need to create these projects so basically as a   fresher you don't have anything to show showcase  as such right you don't have any work experience   that you can showcase to your Employers in  the job interviews so in order to show these   employers that yes we know these tools we have  done some Hands-On on it we have some sort of   experience with it we make these projects okay  so these projects are not very complex projects   but they are simple projects which give you  a brief context of what you will be doing as   a devops engineer okay I would suggest to keep  making more and more projects and thus exploring   multiple tools before you actually sit for an  interview okay that would be more than enough   okay so first of all um if you don't know what  is an eks cluster you must know like like what   are we actually deploying so basically this  particular eks that is elastic kubernetes service so basically this particular service  is a managed service which is provided by AWS yes so why is it called as a manage service   now if you know about kubernetes then  you must be knowing that kubernetes   has multiple components and the two main  components of which are like the control plane and the worker nodes right so basically control plane  is something that keep keeps the control   of the overall cluster okay and there  are multiple worker noes on which your   applications would be actually running okay  so this is The Mastermind that is control   plane that is going to control everything  in your cluster now this control plane is   something that is managed by AWS in case  of an eks cluster okay and and we as a   customer or users not users as a customer  we are going to uh create and manage these nodes are you getting my point so these are  the things because of which it is called as   a managed service so you don't have to create this  control PL plane from scratch right you don't have   to configure everything every little thing from  starting from scratch you don't have to do that   so that is something that is managed with the help  of eks okay so there are multiple such managed   Services provided by different Cloud platforms so  basically if you consider aure then they have this   AKs that is azure kubernetes service then in case  of gcp we have got GK that is Google kubernetes   engine right so these are the things uh which are  the managed Services which are kubernetes Services   right so I hope what is eks is a bit clear if  not then you must first try to understand and   the kubernetes right so the prerequisite for this  lecture is that you must know about kubernetes as   well then only you will be able to understand what  all things are actually happening over here okay   so that's the thing that you need to learn first  now the second thing is now what is terraform   right so if you don't know anything about  terraform you can watch my playlist I have made a   special playlist for terraform but uh if you want  to know in short or in brief then this particular   terraform is an IAC tool now what does this IAC  mean IAC means infra as a code that is infra structure as a code tool means what like  if we want to create an easy to instance   or any such resource on cloud platform like  AWS then in that case we directly create it   right but if we want to create these particular  Resources with the help of a code then then in   that case we make use of terraform and why do  we use them what are its benefits everything   is explained in my introduction uh video  wherein I have explained all of these in   detail okay so it is just like we are making  use of this code to deploy our infra so this   is what you need to understand overall right  so this is about terraform now let's try to   understand what is is genkins so basically  genkins is nothing but a cicd tool right now   what is a cicd tool so basically in a software  development life cycle that is sdlc so in that   case we have got multiple stages wherein our  code goes through that is we first write the   code then we build it right then we test  it right then we deploy it so these are   the multiple stages of software development life  cycle so in this case all of these processes are automated okay so all of these are automated with  the help of genkins cicd pipeline so we make use   of genkins for that but in this particular  case we are going to make use of genkins to   kind of validate test our terraform code and  then deploy the changes to our AWS with the   help of this particular terraform right so this  is what is going to be done by Jenkins okay so   this is pretty much about what all things  we are actually going to implement right now let's try to understand what is the cicd  pipeline that we are going to implement in   this project so basically you can see that first  we develop the terraform code okay so once the   terraform form code is ready we will be pushing  this particular code to the GitHub repository   right so then this GitHub repository is going  to trigger a genkins pipeline now this genkins   pipeline is something that is going to deploy the  changes to the AWS Cloud platform thus creating   an eks cluster with the help of terraform so all  these things are going to happen uh one by one   and in sequence okay so this is pretty much about  our Pipeline and it's a very simple pipeline so   most of the people are charging so much to teach  these projects so please go through this project   carefully so that you can take the benefit of  this particular lecture okay so let's try to code   but before that let's also so try to understand  the prerequisites of it okay so there are some prerequisites so the prerequisites are  first you must have an understanding of terraform you must also have an understanding of kubernetes okay you must also have an AWS account then after that you must  also have terraform installed on your machine fifth thing you must also have got the  access keys of your account okay so you don't   need to have genkins installed on your machine  or anything other than that you just need to   have an AWS account terraform installation  done and you must have the user access keys   with whichever user you are implementing you  must just go to the console just go into the   security credentials and within that you will  get these access keys if you don't have you   can just create it okay this is really simple  what are we actually going to do okay so we   are going to create an ec2 instance so this is  our ec2 instance right and then we are going   to deploy genkins on it okay so this process is  also something which is manual but we are going   to do it with the help of terraform itself okay  so this is one thing that we we are going to do   then after that we are going to uh create  an eks cluster and who is going to create   that for us this particular genkins is going to  trigger a pipeline and create an eks cluster for   us okay so this is what is actually going to  happen so first we are going to create an ec2 instance first thing we are going to create an ec2 instance plus deploy Jenkins on it so  this is the first half of our project   and the second half is that we are going  to create or I would say is to just a minute and the second part of the project  is to write terraform code for an eks cluster now our terraform code is ready then  what we are going to do is push the code on GitHub then the fourth thing is  we are going to create a genkins pipeline okay which is going to deploy  an e cluster now once the pipeline is ready then we are going to just deploy the chain changes to AWS right so first  we are going to create an eks   cluster and then after creating the  eks cluster we are then going to implement a deployment and then we are going to implement the  deployment file with the help of cube   CTL which will deploy an engine X application  on our eks cluster and we'll be accessing that particular that particular application with the  help of load balancer okay so that that's it now   let's go and start coding so basically what I  have done is I have created a new repository   named terraform genkins eks and now I will be  cloning this particular repository on my local   so I will be opening the git bash over here  okay and just cloning this repository with   the help of git CL and paste so now new folder  has been created over here okay so let's try   to go into that particular folder okay and in  this particular folder we are just going to   open the vs code okay so we are done with this  now before actually starting I would let you   know that we won't be creating any modules in  this particular video we are going to reuse the   already existing terraform modules okay now this  is a New Concept so pay attention throughout the   video so that you get a clear picture like what  all things are we actually using so uh first of   all I would like to create all the files and  then we'll jump on to this modules concept okay so now let's try start with the provider  first okay so so first of all let's mention   the provider and it is AWS of course so uh before  actually applying the changes you must have this   AWS CLI configured okay so you have to just uh  paste your access key ID as well as secret access   key ID as it is into this then Define the region  name in which you are working in okay and then   just uh you can keep the default format you don't  need to change anything over here at enter so in   this way you have configured your aw CLA So within  the region we are going to mention the region name   that we are going to use then after that in the  back ends. TF we are going to make use of a remote   back end that is S3 okay so you must already have  an S3 bucket already created created so I have got   this particular bucket already created over here  so I'll be just uh making use of this bucket so   I'll just go over here and mention the terraform  block right here and then after that I'll be   mentioning the back end the back end is of course  S3 and then I will be just kind of um mentioning   the bucket name right here so the value for the  bucket should be taken from uh this particular UI   itself okay and you'll be just pasting it right  here then after that you will be mentioning the key okay now key is something where you store  your TF State file right so in this I will be   just trying to segregate it for genkins and so  I have mentioned it as genkins terraform dotf   State okay then after that I will be mentioning  the region which is Us East one once again okay   and make sure that you don't make use of  variables in this particular block it is   recommended to actually uh add the values  here self okay that's it now then after   this we are going to configure the data  so this uh data sources we are going to   make use for fetching the Amis right so Amis  are already present you just need to fetch   them right so for that I will be just going  through AWS Ami data source terap form okay   I will be getting one of the examples and  I'll be just making use of it see you got this okay yeah I would be make modifying  certain things from here okay so don't make   use of executable users as self if you are uh  making use of owners as Amazon okay okay so I   have got one uh Ami filter so I'll be making  use of that over here okay that's it from my   side and then after that I'll be uh creating  one more data source for AWS availability Zone okay and in here I would I would be just  mentioning the A's fine okay now let's go   to the main.tf and in here the first thing that we  are going to create is a VPC the second thing that   we are going to create is an SG that is security  group and then after that we are going to create   an ec2 instance quite simple so in this we won't  be directly uh I mean creating any modules and   using them here what I'll be doing is I'm going to  make use of already existing modules so let's type terraform let's say VPC modules and in here you  can click on the first link that comes up and   uh you get an example right here okay so what you  have to do is just grab this particular thing and   just kind of use it in this okay so what we are  actually doing is we have got this terraform AWS   modules section So within this we have got all  the modules required for a different resources   okay so we are just making use of those modules in  our particular code okay just type it as genkins   VPC for the cidr we are going to make use of  a variable so let's go to the variable section VPC uncore cidr and in here we are going to  mention the description which is VPC c r and   the type over here is of string okay that's it  it's simple and quite easy as well and within   the terraform tfrs you are just going to paste  its value which is 10.0.0.0 sl16 right just save   it and in here you are just going to pass it  as where. vcore CID that's it within the EES   they have already hardcoded this but what we  have done is we have made use of data okay and   we are going to make use of this particular  attribute that is names fine and we won't be   making use of private subnets we are going  to go with the public subnets and only one   subnet is sufficient for us uh since we are  going to just create an ec2 instance right   so we are going to uh create a variable  for this so variable is uh public undor subnet okay then the description  is here we have to mention the subnets cidr and the type is list of string okay now  now once you have defined this just go over   here for the public subnets you will just have  to mention the value I will be mentioning 1.0 sl6 just just one subnet and that's  fine okay v. public subnet done now   I would be removing these I  would rather go for DNS host   names to True okay and this fine uh but  I would also like to have the name that is Jenkins hyphen VPC okay so this much  is sufficient to create a VPC the rest of   the things uh that is internet gateways then  the route tables the route table associations   everything is done for us with this particular  module you don't have to Define any of those   things okay it gets automatically created  for us okay so let's try to uh just test this terraform U let me just do terraform fmt  I will be initializing the code first of all you see this particular line that is  downloading this particular module from   registry. terraform doio just the difference is  that terraform is managing these modules okay   and the previously created modules were  created by us that's the only difference okay now let's try to validate the changes okay now the configuration  is valid let's try to plan the changes plan is basically to  preview the changes that's all okay it's not a valid just  just a minute okay I have made   a mistake over here yes so it should be uh 24 over okay so like it's shown me everything that it  is going to add okay so this is the VPC that   it is creating it has also created the subnet  the route table Association then you can see   the route table as well along with the AWS route  also then the AWS internet gateway this default   Security Group is also being created okay  okay so now we can just apply the changes okay and it is going to create a VPC for us okay now the changes have been applied  let's go to the console to check uh whether   the VPC got created or not so over here  you can see that Jenkins hyphen VPC got   created then we have got a subnet that is  this one okay so the same tag is visible   for that as well right so instead of  um giving a single tag to all of them   we can also make use of like public subnet  tags so that tag will be specific for that   particular uh subnet so subnet tags and just  mention the tag name as Jenkins hyphen subnet okay uh now it will be just modifying few changes okay so the changes have been deployed  now let's go and refresh it yeah over here you   can see genkins subnet right here then this  is the genkins VPC now this this is the route   table okay that has been uh attached to this  particular subnet as well okay then we have got   this particular internet gateway as well so all  of these things are getting created automatically   okay so you don't need to worry about it so  that's it now let's jump on to the other things   that is about creating a a security group so  let's go to the terraform module of Security Group yes we have this over here just go over here so in this way we are going  to Define our Security Group okay so let's try to remove this description which   is already there defined okay  so let's name it as Jenkins SG and the description is Security Group for Jenkins server fine now the VPC  ID will be now this VPC ID will be coming   from this particular module right so we  would be defining it as module. VPC dot VPC underscore ID okay so we are going to make use   of this fine so this will return the  VPC ID which will be attached to our   security group just mention it as SG  I would rather prefer not to mention these I will be just mentioning the 8081 okay so  this one is specifically for the http and the CER   block is 0.0.0.0 sl0 since we want to allow it  from all the uh IP address ranges okay again I   will be just copy pasting this part once  again and in here I will be making use of SSH okay and again it should be like um from everywhere so  these two are what I need then uh I think we must   also have got an eess one yeah so for that we will  have to go to the input section and search for the   egress cidr blocks so this one you can see over  here but I don't want just the cidr block I want   egress with C block so this is what is the list  that we want to mention okay so this is another list then protocol is equal to -1 and cidr blocks will be slash Z okay so in this  way we have created a security group yes so now let us give it a tag that is the name which is equal to Jenkins SG just save this okay and now  we'll go to our get bash once again   and try to just format the things  again and just try to validate the stuff okay so first we need to  initialize it right because the   module needs to be downloaded first  before we actually use it so that is   done only with the help of init Command  right so terraform plan now let's first validate okay the configuration  is valid now terraform plan we will be applying the changes okay so now the apply has been  completed so let's go to our uh ec2 and   and let's check whether the security group has  been created or not yes the security group has   been created over here let's check whether  it has got the desired inbound rules as well   as outbound rules so yes we have got 880 so  basically we are opening this 880 Port range   just for the genkin server so that we can  install uh the genkins on this particular   thing and 22 is basically for the SSH that  is secur sh okay then after that we have   to create an ec2 instance so now let's go to  this terraform AWS ac2 modules and this is the first link wherein we can get this yes so  over here you can see these are uh certain   examples so since I have to create a single e to  instance I'll be just grabbing this particular   code from here and just using it so now it  is easy to instance so the name I would like   to mention it as Jenkins server okay and for  the instance type of course uh we'll have to   create a variable for it okay so uh for the  key name also we'll need to check which key   names are available for us and also the  security group ID as well and the subnet   ID as well okay and also one more thing is to  associate the public IP address okay so that   is also something that we need to address  but let's go with the instance type first okay and just try to mention the name  over here as well and the name over here is Jenkins server okay now let's code  to the variables and Define the variable as instance type instance type and then  we have got this type as   string okay and now let's go to the terraform tfws and just mention the instance type as t2. micro fine and within this uh we are just going  to go over here and mention its instance type   here itself okay let's go to our console once  and let's try to see the key pairs which are   available so Jenkin server key is available  so I will just copy this particular key and   this is what we are going to use in here as  well okay so keep monitoring as so no worries   with that now the security group that we want  to use is this one right so it's of course the   module SG let's mention it as module do  SG dot now what is the output of it that   is something that we need to check so let's go  to the security group and in here let's go to   the okay so there is this do security group ID  so this is something that we need to get right   this Security Group ID and again we have uh we  need a subnet ID of course right so the subnet   ID would be coming from VPC itself right so  we are creating a subnet with this particular VPC we will get these public subnets from here  okay so these are this is going to return the   list so how can we get the value is by just uh  mentioning it as the first value okay so just public you can see public submits over here and  just mention the first one okay so yeah now other   than that what if I want to associate public  IP address as well so then let's go back to the inputs public IP yes so this one is  the right one that we want for our C2   instance okay once we did that now we  need to use a user data so we are not   going to manually install the genkins right  so what are we going to do so we have got a   particular we have got this commands that  we need to follow or execute on our ec2   instance but what am I going to do is I'm  just going to create a file let's say it as Jenkins install.sh so basically this is the file that  contains all the commands and this will be   pushed as a user data for our ec2 instance  if you don't know what an user data is uh   user data is something that executes or it's an  file that gets execute when your ec2 instance   is launched and starts running so it is uh  executed then okay so let's just grab these commands Okay and just paste it right here  so what all things are you going to install   you are going to install genkins you are  going to install G you are also going to   install terraform you are also going  to install Cube C okay so these are   the things that you are going to install  okay just save this particular file okay   now within the main.tf within the user data  section you are going to pass the file named Jenkins hyphen install.sh now this is  going to become our user data okay uh   that's fine then we have got availability  Zone as well now which availability Zone   you want to work in right so definitely we  have defined the data section and within   that we have got names and that to the  first name is something that we want to   make use of now let's go to our get bash  and try to apply the changes terraform in it great now terraform fmt that is format it is  going to format my files so all these things are   done manually over here right so these things  are going to be automated with the genkins so   how that is something that we are going to  learn okay no need to worry so terraform validate okay there is some error module. SG is a  object and this object does not have an attribute okay Google is your best friend  remember that search whatever   error you get this object this object  does not have an attribute named this okay this issue was with a wrong reference  to the remote State I was referring to the   different remote State and that was not having  ALB attribute after going through the code again I realized fine okay this is good working as a Security  Group uncore ID okay so let me try with this yes it's working with Security uncore Group  uncore ID so sometimes documentation is also not   the right resource to refer so you must Google  it and find out what all things we need to do so there's only one thing to add and  that is our genin server so now let's   try to apply the changes hyphen hyphen Auto hyphen approve belong to different networks okay  let me check one documentation of VPC okay and in this let me check about the VPC ID so there is one more okay so VPC  ID is what is desired over here   and not the default VPC ID  okay so yeah now it should work okay okay now we can see that five  resources have been added now let's go to   our console once okay let's go to the instances  section and we can see a jenin server okay in a   running State and it must have a public IP  address as well it does and it must have a   security group associated with it which which  has got two Port range in this inbound and for   the outbound it's allowing all of them yes so  everything is configured exactly how it is it   should be uh I think everything is done  properly yeah so now I mean let's try to   access this genkin server okay just copy this  and kind of paste it over here and just um run 88 okay so there may be some sort of  problem so basically the problem with   the code was that we hadn't mentioned  map public IP on launch in the VPC we   missed this particular shebang for the  shell script file so I'm adding this   over here now let's see if it works  for us okay I will be destroying everything okay we are done  with the Destruction part   then now once again we'll be applying the changes okay so the now the apply has been  completed so let's try to refresh it once now let's check it yes so now the genkins has been installed  and you can see this UI if your genkins has   been successfully installed Okay so so once  your genkins is installed you can just go to   this particular path on your E2 machine so just  go and connect it okay let me connect and show   you once again so just click on this server  and connect it and again go to this connect option okay then just kind of mention sud sudo cat because we have got  that initial password at this particular   location and you get this password okay  so you just have to copy this password   and just go to your uh UI and just paste  it and just continue don't save this yeah   install the uh suggested plugins and  it is going to install plugins for you so just enter the details so I'll  be just entering few details out here okay cool so this will be my  genkin save and finish start using genkins so now this is the main dashboard that  will be visible to you once you have done all   the uh steps that I have mentioned okay you  will get this particular um dashboard but we   won't be jumping in right here first thing  first we have to create an eks cluster code   first right so let's go to our VSS code once  again and don't forget to mention the file   that is dogit ignore see these files like  terraform folder as well as this terraform   dolog docl so these files are not recommended  to be pushed to our um GitHub repository so   better is to keep them in get ignore so that  they won't be um getting added so just save this yes and you would be fine  now I'll be closing this uh vs   code as of now and I'll be opening it in  another folder oh my God I just created   a single folder okay no problem so I will  be creating a folder specifically for the ec2 with genkins or just say Jenkins server okay and just include all of  these files okay in this particular   folder that is genkin server and now  we are going to create one more folder   for our eks cluster okay so let's  open this one with code now Okay so okay we'll be mentioning the provider over here so   let's try to mention the provider  that is AWS and the region again Is Us East iPhone 1 contrl s and within  the back end we are going to define the   back end as S3 once again and  the same S3 bucket I would be considering let me once again  go and copy that particular this thing and the key over here is like uh eks  SL terraform dotf State then the region is   Us East one fine now okay so rest of the things  will be adding as and when required so uh first   of all the other thing that we require is to uh  Define something and what needs to be defined   is a VPC once again we are going to create  another VPC and within that we are going to   deploy an eks cluster so what I will be doing  is let me open or else what I can do is I'll   open both of these folders and the sing SLE  vs code yes okay cool so over here within the   genin server we have got this main.tf wherein  we have defined the VPC so I will be just uh   including the same U module as it is I will  be just uh making the changes in the VPC CID okay so variables also I can take it from here itself both of these whatever is repetitive you can just copy and   paste it okay so that's fine now within  the terraform dotf bars just copy this part and just paste it over here and instead   of this particular uh range we  can make use of 192 168 right and just kind of replace the first two things  with this one okay so that's pretty much about it okay uh um we'll be just um editing these names uh I think   I would have to change everything in  the tag section so let me do that first okay okay so now we are going to modify  a few things in here now we also need   group of private subnets as well so  we are going to actually deploy these   this eks cluster in a private subnet okay  so let's go to the variable section and add just kind of copy this stuff and  just name it as private and save it   okay so then over here you are just going to  mention it and you are going to make use of   multiple subnets over here multiple ranges  I would say yeah so let let's give them the ranges so let's start from  one only one then there's two and lastly we also have got the third one okay so here you have got fifth one and  the sixth one done so now we are going to   add this subnets in our uh this main.tf okay  and over here yeah it's already mentioned so   no need to mention it once again so few  things are going to be same so I'm going   to remove this map public on IP on launch  as well and uh let me also enable the natat Gateway why because we are making use  of private subnets over here right so   we need them we also are going to make  use of single net Gateway okay fine now   after this we are going to make use of  different tags out here okay so I have   got a list of tag tags that we need to use  over here so basically you just have to make   use of this kubernetes doio cluster SL  my eks cluster is equal to Shared okay   just copy it over here yeah then after that  similarly for the public subnet tags as well it has to be there and one more uh  tag we'll be adding right here that is eb1 now there are private subnet tags as well yes here also these two things are going  to be same so just the differences of the   instead of elb it is going to mention  internal elb that's it okay so now our   VPC is ready okay now let's go to our eks  module right so now this is something new   right so let's go to our documentation and just  search for terraform eks module okay just go   over here and you will get an example of course  written right in front of you so you can grab it okay just take it from here and just paste it  over here okay now there are certain things   that you need to remove from here so I will be  defining it as my eks cluster so the version   that I want to use is 1.24 uh I won't be  making use of any cluster add-ons or such   something like that so VPC will come from  the above module that is module do VPC do   uh VPC uncore ID yes and this particular  uh sub sub Nets should come from private   subnets because we want to deploy our eks  in a private uh this thing right so VPC do private subnets okay so it would be  fine and within the eks managed uh   node group defaults um I don't think  so so I would be uh making use of it   just the eks node groups is fine and  within this I would be defining a node okay so the things that we mention  over here is so basically this is   the node group I would say yes and  the name that I have given over   here is say nodes okay now the group  must have minimum of one node maximum of three nodes or any number you think is required  in your case then this desired size okay now this   is the requirement that is it should have  two and Max it can go up to three okay so   this is according to our requirement and it  would change with a business need right so   that is the thing fine and then within this we  are also going to define the instance type as well and the instance type should be written in this  particular list format let's   keep it as t2. small okay and that's  pretty much what is required I think okay so yeah okay so this is done almost so let  let me save it now let us go to our get bash and let me enter this  particular folder that is eks okay and I'll be just  initializing the terraform code I'm not going to deploy it using uh like  this method only it has to be done with   the help of genkins itself but I'm just  trying to check whether everything is   right or not so terraform uh fmt let  me validate it also terraform validate okay so this has not been declared  in the root module okay so have I   not oh I'm so sorry I didn't  Define it and I'm using it wow wow just Define it over here and you are done okay so now it is pretty much done now let me plan it cool so everything is uh working fine  so what I'm going to do now is I'm going   to push this code to my Repository okay  so let me come out of it and before that   I would also like to mention the git ignore  as well so yeah new file just mention dotg ignore okay just save it yes now let's go to our get  bash once again and now let me check the status first okay now let me once again check this get status yes so  get add we can add these changes get commit hyphen M adding files repo that's it then get push hyphen U origin and whatever the name of the branch is right so now let's check GitHub first right within the repositories let me  check yes so these two changes   have been updated right here okay now  once we are done with this uh we will   be just creating a pipeline in our genkins  okay so within the new item I'm going to   select the name as say terraform C eks  cicd here so this one and Pipeline and okay okay so this is pretty good now first things  first is you will have to add the access Keys as   well as Secret taxes Keys now why see because  uh Jenkins is going to make changes to your AWS   right now for that Jenkins need to know the  credentials the authentication needs to be   happened right without that it can't make changes  to any other platform so for that matter we first   need to add credentials so how do we do that  so let's go to the dashboard okay in that we   have got this manage genkins in the uh left  side plane okay and within this we have got   this credentials section okay just go to this and  within this click on global go to add credentials   and within this just uh um change this usern name  with password to the secret text why because AWS   access Keys as well as secret access keys are uh  kind of um secret texts so they can't be visible   right so for that matter we have to mention  it that way so the ID for that is AWS access   key ID okay and for the secret we just have to  open that particular file now let me go to my   downloads and search for this particular  file now this one is something that has got my secret access key as well as um access  key ID so so I'll just copy them okay and I'll   be just kind of pasting it over here and create  okay again uh I will be adding another secret   text that is AWS secret access key okay  and I'll be just copying this particular   secret access key and just kind of pasting  it right here and create okay so now I have   created access key ID as well as secret  access key both of them over here fine now let's go to our dashboard once again into  the pipe line and into the configure section   okay so in this we first mention pipeline okay  and agent any and uh we are going to uh mention   the environment details in this particular  section and for the stages we are going to   create one more different section right okay  so in the environment we are going to define   those AWS access key ID okay so basically we  are going to mention it in this way the form   of credentials and the credential name we have  to copy and paste it as it it is it should not   uh be slightly different also otherwise  it will give you an error so AWS secret access key is equal to credentials okay so we'll be just kind of uh  taking the values so let me open it in a new   tab so over here within the manage genkins  just go to the credential section and grab these just copy this okay and then this one then after that we are going to  mention the region which is Us East   one okay then we also have got the stages so  there are mult we can Define multiple stages   so generally the first stage that  we mention is of course check out SCM okay so within this we are going to  define the step section and within that   we have script section right so in order to  get this script we are going to make use of   pipe line syntax so just open it in the new  tab okay and in here you get multiple options   uh for creating a script right so over  here since I'm going to check out from   the Version Control I'll be just selecting this  checkout and U for the repository URL I will be   just going to my GitHub yes so this is my uh  repository I'll be kind of just copying this   part and going again and pasting it right  here okay since it is public I don't have   to mention the credentials but if if in your case  the GitHub repository or the gitlab repository or   bid bucket repository uh requires credentials  and it is private then in that case you will   have to create or add those credentials over  here okay so it's simple and instr instead of   Master I will be mentioning it as main okay so  just generate the pipeline script and I will   be just kind of copying this part and kind of  pasting it as it is okay so with this my first   stage will be getting completed so just try to  save it and let's check if it's building for us for viewing the console logs you can just click on  that uh three dots and you can see that checkout   is successful okay so again let's go to the  pipeline part and again let's go to the configure section now in this we are going to add  multiple stages like uh for the terraform   we make use of multiple steps right so  first we init that is we initialize it   then after that we format it we validate it  right so many things we do in pipeline so   similarly we have to mention those stages  here as well okay so the first stage is initializing terraform okay so let's try to just say kind of within the  steps section we again have the script part and   in this we are going to define the directory  within which we have to work in so basically   the directory that I want my terraform in it  to work in is of course this eks right so I   will be just mentioning the eks a directory  path over here so just mention it as eks and   within this just make use of sh and make use  of terraform in it okay so this is our first   step of terraform that is initialized in so  let's save it and let's check if it's working properly so it's initializing you can see okay so it's successful now let's go back now in this again we are going  to just copy this particular stage   once again and we are going to  uh use it for formatting the terraforms so terraform fmt  similarly for terraform validate as well formatting terraform code so terraform validate okay so simp simple and  then after that after validating   we of course do terraform plan so you can also say previewing the infra using terraform so just say terraform form plan okay that's it just save it and build now so basically it is validating okay so there is success and now  it will show me the terraform plan details as well okay so all the changes have been shown  right here okay so you just mentioned few   things but under the um underlying  so many things are happening you know okay so that's great you have  to add 55 more things and yes so now uh so after this particular stage we are going  to add another stage of applying the changes or we can also say creating and eks cluster okay steps then within that we have got script the  directory would be the same one right that is eks and within this one we are going to  mention terraform apply hyphen hyphen   Auto hyphen approve um just let  me save it and now let's try to build see most of you must have got confused  like from where this terraform came we to   haven't even installed but if you remember  in the code while installing the genkins   we also installed this particular terraform  along with that we also installed this Cube   CDL as well so you don't have to um think  like from where this terraform came so this   terraform came because we had installed  it while creating the ec2 instance itself okay and all always remember whenever you  are running a pipeline okay so you can't   put um just terraform apply you must have  to make use of hyphen hyphen Auto hyphen   approve because otherwise it will ask you to  enter uh to confirm right it will prompt to   confirm which is not possible in case of a  pipeline right so that is something that we   need to be aware aare of and we need to uh take  actions accordingly so all the things are getting   created which means that the authentication  has worked properly for our Cloud platform okay so this e cluster takes a lot of time to create so  now I'll be pausing this video and just resuming   it once it's um it gets created now you can  see that the apply has been completed and all   the resources have been created right so this is  how we are implementing the genkins pipeline but   um in ordinary cases like if I go and apply the  changes using terraform then I get a prompt for   confirming whether I really want to proceed or  not but here I am not getting that option right   so if I want to add that particular option here  also then what I can do is I can do it in it in   some other way now how can I do it so I will be  just making use of input function now with the   help of this function this particular genins  stage will ask me like whether you want to   proceed or not okay if I say proceed then only it  is going to execute the rest of the pipeline else   it is going to abort the pipeline okay simple so  just we are going to add a message over here that is are you sure to proceed okay and then if it's okay then you have to say proceed okay  so this is what is the input message   that I would like to add simple let me  just save this and now we are going to   again build the pipeline just to show you  how this input is actually going to work okay now it has done now when you hover your  cursor over this you will get this option like are   you sure to proceed so if I say proceed then only  it is going to continue with this pipeline I'm   going to click on this and now it has proceeded  for creating an eks cluster so unless and until I   don't go for proceed it is not going to continue  with the pipeline so this is the benefit or an   additional thing that we can customize for our  pipeline um according to our requirements right   now okay well and good that we applied the changes  we created and eks cluster that's definitely good   but what if I want to destroy this cluster that  is the infra that whatever I have created I just   want to destroy it then what right so in that  case what we are going to do is I'm not going   to destroy as such but just uh adding an option or  a feature I would say so in this we have got this   project is parameter so in this case I'm going  to add one parameter that is a choice parameter   so it is going to ask me which one to select okay  so the name would be like action and I'm going to   just give it two options like apply and Destroy  right so these are the the action to be performed by terraform simple so this is how we  are going to add the changes okay it's   pretty simple and we are done yes  so in this way in this particular   creating creating or destroying I would  say depending on the uh choice that you make right so this action is going to be  considered for that respective task okay simple   is it now you have configured your eks cluster  now the next stage that I'm going to add in is   to create a deployment as well as a service on  that particular eks cluster so so I have already   got these files I'm just going to add these files  to my uh eks okay so um but this is for the infra   right so instead of adding it here itself what  I can do is I can just create another folder so let me do it in my local only  so that it would be more conven   venient for me as well within the  projects I've got this project and   this eeks fine so let's try to create a  new folder for this specifically for the configuration files okay so within this I'm going to uh kind  of create a new file that is deployment do yaml and one more file would be um service Dot yaml right so what is deployment what is service  you must be knowing if you know how kubernetes   works right so what I have done is I have already  created this files over here I would just like to   explain it in short like what all uh things it  is doing so basically the API version is apps uh   slv1 then the kind of the SP particular um file  is deployment or the resource is deployment then   we have got a metadata and we have just named  this particular U deployment as engine then   within the specification section we have created  a template right and we have also created uh or   mentioned the metadata wherein the labels  are app equals to engine X within the spec   we have mentioned the containers so basically  kubernetes is used to manage these containers   okay so what are containers and everything that  we are going to create a new playlist for it and   we'll be covering everything in there but for  time being just try to understand a package   of containers Okay so package of application  sorry not containers it's just the package of   application and its dependencies simple okay  so you are actually converting or packaging   your applications in the form of containers and  the name of this container is enginex the image   is enginex so basically when you mention like  this it is going to pull that image from from   the dockerhub repository okay and then within  the ports we have this container Port as 80 and   we are going to save it then for the service what  we are going to do is we are just going to select it okay so in this we have got API version as V1  then the services the kind is service and then   we have got this metadata again similar to that  of the previous one labels are same right the   ports are like 80 why because the container  Port was also 80 so we need to match it and   pass it right so the Target Port the Target  Port is 80 protocol is TCP the selector again   is app enginex and the type is load balancer so  there are three different types of services that   is uh cluster IP then we have got node port and  this third one is nothing but load balancer okay   so it is going to create a load balancer for us  okay so let me save all of it and now let me go   to the get bash and we are going to push these  changes to are GitHub report so committing the changes adding configuration files okay  then get push hyphen U origin main so now we have pushed the changes  to the repo Let me refresh it okay within   the eks you can see this configuration  file section and we have got these two   uh files that we are going to apply with  the help of cube C right so basically we   are going to make use of cube CDL  over here in our pipeline so stage appying engine X application okay we are done with this  then the steps we just need to mention the script and within this we are going  to mention the D that is directory   that is eks slash don't take risk  just copy it from here itself and   copy it properly okay and then after  that you'll be just pasting it right here then what are we going to  do is we are going to make use of AWS there is a command  specifically that is AWS eks update Cube config hyphen hyphen name my whatever the name  that you want to mention you are going to do   this now we can see that AWS eks update Cube  config name over here right let me just save it okay so there was an error so you must specify  a region uh you can also configure your region   by running a WS configure so we also need  to define the region out here so there is   some change that I need to make in here so  it should be default region that's why it's   not taking okay so the environment  variables must be precise as I told you build yes so that was the only problem okay so now what  we are going to do is we are going to just apply   Cube CTL okay since we have already installed  Cube CTL we won't get any problem over here okay   hyphen f and the path that we want to mention  that is deployment. yaml okay so this is the command that we need to implement apply hyphen f  and whatever the service do yamalis okay now just   save so you we can see that this is the E cluster  that has been uh created already now the only   problem that we are facing over here is we are not  able to apply the changes right and the error is this in context of like just let me mention for eks the IP used is a private it cannot be  accessed outside so change your clusters   API server endpoint access to public yes  so something that I missed okay is that   within this particular main.tf I had  to mention something that is cluster   endpoint public access to be mentioned  to be true okay so now let's try to add it commit the changes okay now let's try to push it now on pushing it I think we didn't enable the polling  right so that is something that we can   enable right here okay so let go let  me go to the configure section and   within this you can just mention poll  SCM and mention the five stars over here okay but now I think uh we'll have to make the changes now itself okay so  where is this yeah let's build it with   par parameters once again and now  again it is going to take a lot of time for e okay so it is out of memory once again okay so again I will  have to once again configure it let us try to configure build with parameters once again for for I think it's again out of memory guys so what  I can do now is I can either just change it   in instance type go to the terraform dotf  and mention it as t2. medium and just save   it now let's go to the CD e or just LS  CD genen server and terraform apply High okay now the modifications have been done  let's go to our um instances and let's   just kind of refresh it or it's just kind of  initializing so let us wait for it to initialize first so let's try to build build so let's proceed hey guys now you can see that this deployment.   apps as well as s service has been  created so now let's go to our load balancer you can see that a load balancer service  has been created over here let us wait for it to   be active so now you can see that it shows welcome  to engine X which is because of the deployment   file that we deployed on our Kubernetes cluster  so I hope you found this video helpful please like share and uh also there is one more thing that  if you want to destroy this pipeline what you   can do is you can simply go to this particular  terraform and within the build with parameters   you can simply say destroy and uh click  on build it is going to destroy the whole   infra that you created using this particular  pipeline so that's it for this thank you so

Channel: Code with Gauri

Views: 26,605

Keywords: devops, project, terraform project, terraform, eks, jenkins, cicd, complete devops project, Terraform and Jenkins, EKS Cluster, AWS, IaC

Id: wY8VFIAz_Og

Length: 118min 16sec (7096 seconds)

Published: Wed Oct 18 2023