DevOps Real-time Project #1- Deploy AKS Cluster in Azure With Terraform

when it comes to infrastructure provisioning terraform is the market leader there is no route about that be it AWS gcp Azure or any other cloud provider for that matter of fact terraform is the tool of choice nowadays but when we browse the internet we mostly see AWS related videos and blogs and only a handful of content related to azure that is why I am publishing this video for you as an attempt to fill that void hello everyone my name is piyush sachdeva and in this video you will learn how to set up AKs cluster in Azure using terraform we will be creating a resource Group and many custom modules for resources such as service principle Azure keyword and AKs cluster so please stay with me till the end of this video and I promise you will not regret this [Music] you can see I am using visual studio code to create all my terraform configuration and files you could use a ID of your choice that you are comfortable with but I am using vs code for this purpose so the first step is to make sure you have all these extensions installed basically we need these three extensions first is azure CLI to interact with azure make sure it is installed then the second one is azure account which will help you manage your Azure subscription and your login details integrated with the IDE and the third one is your terraform extension provided by hashicor make sure you use the official one that is provided by hashicorp all right after you have installed all three extensions let's go back I have created a new folder with the name TF demo I'll create a new file by clicking over here and name it main.tf so this will be our root module and as part of this video we'll be creating four different resource types in Azure using terraform the first one is your resource Group then we will be creating a service principle a key Vault to store the service principle secret credentials and AKs Azure kubernetes service so as part of the first step let's initialize provider over here I'll write provider Azure RM and you can initialize with an empty block with just features in it this is a mandatory step that you have to do otherwise it will throw the error next step is to create the resource Group I will not be creating a custom module for the resource Group I will be doing it in the main module itself in the root module but I will be creating custom module for rest of the resources so please be with me till the end of the video so to create the resource Group you create a resource let's type Azure RM Resource Group and you give it a name let's call it rg1 inside this block you provide your variables and initialize those variables so the first one is Resource Group name I'll write name over here and the second one would be Resource Group location so location it now we need the value of these two variables but before we input the values we provide the values we have to declare these variables so I'll create one more file in the root folder itself call it variables.tf and I'll create two variables variable variable name was RG name then you can provide a type call it type as string then optionally you can provide a description as well let's call it Resource Group name and the other variable was location provide the type of it which is string again and you can alternatively provide a default value as well so let's say I would want my Resource Group to be provisioned in Canada central region if it is not provided then this default value will be taken in account and again description is optional I just skip it for now right now that we have declared our variables over here now we can use it in our main.tf by VAR dot variable name which is RG name and where dot location so this is done now we can pass the values to these variables using different methods you could use many methods that have been been provided by terraform I'll put the link in the description section below I'm gonna use a terraform.tf war files so I'll create a new file over here name it terraform.tf War so that it will be loaded when you run the data form plan and apply and it will provide the values to those variables a variable name was RG name and the value I can give is test views RG the next one was location I can provide a default location such as Canada Central that's it now I'm gonna go ahead and open my terminal window I'll use git Bash all right I'm inside my directory where my terraform code is there so I'll just create an alias terraform okay now I'll run terraform in it it will initialize the back end says terraform has been successfully initialized there are no errors now I can run TF plan to see the suggested changes okay it says invalid resource type Azure RM Resource Group because there was this capital r quickly go ahead and correct this right I'll run it again okay it provided the output it says plan one to add 0 to change and 0 to destroy that means one resource will be added zero resource will be destroyed and zero resource will be changed as part of this terraform plan location is this that's what we'll be using the name would be this and ID will be generated was the resource Group is created right so this looks good now we can run terraform apply iPhone I find auto approved so that it will not prompt us for the confirmation all right it says creating and creation complete so let's go to Azure portal and see if the resource has already been provisioned now I'll go to Resource groups and you see my Resource Group has been created test use RG doesn't have any Resource as of now so this was a pretty quick simple step that we just did now let's go back and see the interesting part which is how to create a custom module right so inside this we can Source the module but before that we need to create it so inside our main directory I'll create a new folder call it modules so inside this module I'll create one folder for service principle and then inside that service principle I'll create two files main.tf and variables.tf inside this main.tf I will write all my terraform code that I don't want anyone to change and the configuration that I want users to modify as per their needs such as service principal name I will specify that in root module I'll show you how we can do that so first let's go ahead and let's search service principle terraform and this is the one I'll provide this link as well in the description section right so I'll copy the first default template that has been provided to us and I'll put that in our module this is the data source which is reading the data from the client config and we will be using that data in the creation of service principle along with the details that we will be providing externally so first is the display name this will be the service principle name but we want user to provide the service principle name so we should not be hard coding this in our module so let's keep it blank for now and we'll update it later then owner will be the object ID of the client config right so it will be taken from data dot Azure ID client config this particular data source the value would be the object ID of current object then we have a resource of type Azure ad service principle so there are two resource types over here one is azure ID application other one is azure ready service principle so when we create a service principle in Azure we first have to create an application in Azure active directory so that this service principle could be registered to that application so this is what it is azure ready application and then we will be creating the service principle and supplying the application ID application ID that's just been created over here and this will be supplied to this particular resource which is azure ready service principle and app role assignment required that means if you could assign a rule to the service principle so default is false but I'll just update it to true so that we can provide a contributor access to this service principle again this service principle will be used to provision AKs so make sure you do this and then again owners will be the same what we have seen like client config current object ID right and this name we will be updating let's update this name as well Azure ad application SPN and I'll use the same reference over here as PN now this field is still pending so we'll create a new variable over here variable service principle name right and this will be of type string and now over here we can specify War dot service principle name right so this module is complete now let's go back to our main.tf and now creating the resource over here would be a little different than what we did uh with this particular resource which is Resource Group because now we are using a custom model that we have just created the service principle so to do that do module and then do a tab name of the module name of the module is service principle and source is the directory where the module exists so that would be module slash service principle over here now you can pass the variables from here which will be replaced in this particular module right so we have only one variable which is service principle name and again we don't want the name to be hard-coded over here as well we'll be providing with the help of variables so let's call it where dot service principle name but it is not visible over here when we press dot ideally it should be like the way location and RG name is there it's not there because we have not declared it in the variables.tfe yet of the root module so let's go ahead variable and name is service principal name type a string and let's just remove this line for the sake of Simplicity and if we go to this now and bar dot now this is visible right so I'll select service principal name once I do that now to create the service principle we have to somewhere pass this value like like we did for Resource Group name and location so again the same way terraform.tf War files provide service principle name and a value let's call it test views SPN so the execution flow of this module will be terraform will detect that it is a module block and it is referenced by a service principal name which looks a local module in our case so it will reference this module and look for the module as per the path specified over here so it will check in the modules directory service principle and load all the files inside there like main.tf and variables.tf and also it will supply this service principle name as the variable to that module and value of this is specified and this file terraform.tfr SPN and then it will be supplied over here in this main.tf so this will be replaced now we can do one more thing if we go back to main.tf now we want our service principle to be created only after a resource Group has been provisioned right so inside that so inside this module what you can do is you can create a depends on depends on and within this square brackets you could mention which resource is this particular module dependent on let's say a resource Group so we can reference that with the name azure RM Resource Group Dot rg1 so this is how we will reference it and terraform will make sure that before creating service principle this is already created now if you go to main.tf you see we have created an Azure ad application we have created a service principle now we have to generate a service principle ID as well so ID is not generated with this and we have to explicitly create one more resource so if we go back to the terraform page and there is this field called Azure ad service principle password there is a separate resource for this so I'll copy this over here paste it The Source tab is azure ready service principle password and it will hold the service principle ID so I'll just keep it the way it is right now once the service principle is created I would want to use some of the configuration of it as part of our next resources for example I would be using client ID and client secret that is generated by service principle I will store those fields in the key vault and also I will use the same in provisioning or AKs cluster right but how do I access values from one module into another module so we will be using output variables over here so to use the output variables I'll create a new file called output.tf and it has the same format like a variables.tf file where you define your type of variable this was the input variable so we just put variable in case of output you specify output and then name of the variable and type and value because input variable doesn't necessarily have a value we pass the value with the other methods but with output variable we provide the value so all the all the variables that we need to use as part of our next module we can specify over here right so these are all the variables that I need service principal name service principle object ID tenant ID application ID client ID and client Secret right so these value will be using later on let's go back to our main.tf now our service principle will be created after this do this let's go to terminal let's do a TF plan okay it says module not installed because we have just created a new module which is service principle first we have to install it to install the module you do TF in it again now the module is installed and we can do a TF plan now okay now let's see now it says plan two to add that means two resources to be created now and these two resources are Azure service principle and Azure ready application so let's do a terraform apply on this like apply completed let's verify your resources let's go to home your active directory and go to app registrations inside that we have an application created with this client ID so that means there is no rule assigned to this service principle so a Next Step would be to assign a role maybe a contributor access to the service principle to do that let's go back to terraform and over here we don't need to create a separate module for that I'll just create a new resource of type azure RM rule assignment give it a name rule SPN and if you want to know the values of it let's go back over here April assignment this is the resource that we need what we need is scope role definition and principle ID so I already have that curated so I'll just paste it over here so this is my subscription ID which uh I have blurred and then the role I'll be assigning to is contributor roles so that this service principle could create AKs as well as it has access to get and put keys to the key vault right and the next step is we need principal ID this is the service principle ID that was just created over here says that service principle ID will be using the output variables that we have used so let me just remove it okay and I'll show you how we can do that so you type module dot the name of the module which is service principle like dot the value of the output variable so these all were the output variables that we have specified so we need service principle ID which is service principle object ID this particular field over here and that is how we will get it and we will also add a depends now over here so that it will be dependent on module dot service principle earlier we did Azure RM Resource Group dot rg1 because we were not using any custom module for the resource Group so we directly access it from the resource itself but over here because we are using service principle module so we have created the depends on module dot solve this principle so I'll just save this if you go to modules output.tf you'll see the service principle object ID and this is the value that we are using and it was generated from Azure ready service principle main.object ID which is in this main dot TF so if you see the values of all these output variables are defined from the resource then name of the resource and the name of the attribute but over here we are using Main and in our main.tf we have specified the name as example and SPN so let me just correct this let's put main everywhere so that it will be in sync now if you see the output variables now it makes sense it will take the resource type then name of the resource attribute so this value will be output as the service principle object ID which we are using in the root module over here as module dot service principle dot service principle object ID right so this is how we use output variables as an input to the root module right so let's create our role assignment now I'll save this file as well TF plan all right now it says 4 to add there'll be additional resources created now one is the service principle password object that was not created earlier because we had some issues in the file which we just fixed and then there'll be the responsible so all those files will be created again because we have made some changes in that so the existing one will be destroyed and newer one will be created so let's do an apply on that all right it is completed let's validate our resource yep so I'll go to Azure active directory and inside that app registration and here is our app that's been created right so let's go back now the next part is to create the keyword so that it will store a service principle client ID and Secret it so again we'll create a new module inside the modules folder create a new folder called key vault and main.tf let's go back to keyword documentation okay so over here we'll be creating secret as well as key Vault so there are these things I'll just copy it and make the changes afterwards so this is again the same Azure RM client config that we use for service principle then so script we don't need this we have already created a resource Group now a key Vault name call it KV and name we will take it from the variable location and Resource Group name again we will be taking it from the variable then we need to provide a tenant ID SQ name let's use a standard SKU soft delete retention days is seven that's good way all right now there are two type of access granted one is role base access another is key Vault access policy so if we use this that we have to explicitly set permissions for Keys secret certificate Etc and the other way is to use this option which is enable our back authorization if we set this to True by default it is false so if we set this to 2 we don't need to mention this okay I'll just delete those and it will grant the role-based access authorization to evolve for example a service principle has contributor access so it will assume that service principle has all the access required to access the key Vault and we don't have to explicitly Set the World level permissions for it right so let's just create a variable files as well variables dot TF the variables that we need over here is is name location Resource Group I'll just quickly create those variables yay that's it now I'll just use those values inside here key Vault name VAR dot location Dot source group name now let's go to our root module main.tf and after this let's create a new module for for key Vault let's give it a name key vault paste some value some value for service principle right so it will Source the keyboard module then use the variable key Vault name which we have not specified yet so let's specify that in the form dot TF War so key for name equal to s piyush KV then location Resource Group name it's already there and we are passing this variable location and all others to the module itself and service principal name is again it's coming from TF War files and service principle object ID and tenant ID are coming from the output variables of service principle it like like we have used for this particular resource creation all right so this is for key Vault let's see if we have everything we have variables main dot TF okay so over here we needed to create one secret as well these were the default values let's update this as well okay so name we would need the service principle secret key and value so module service principle Dot client ID and the value would be the client secret so module Dot service principle dot client Secret right and keyword ID it should be from here as your keyword dot keyword dot ID this will be the ID generated by key vault people as well as here variable key Vault name is a string let's take this secret out from the module and put it in over here now we have to specify the output variable of key vault so let's create one more file put dot DF and output name is keyword ID and value would be Azure RM keyboard Dot dot KV dot ID K and we will use this output variable inside the main dot TF with module dot keyword ID let's do a TF plan now okay it says module is not yet installed so I'll do a TF in it and then TF plan says unsupported argument in module so I'm getting error for these three variables over here and it says unsupported argument so maybe because I forget to specify those values in the variables.tf of the this module so let me just add the values over here so service principle name object ID and tenant ID and let's do a TF plan now hopefully this should work now all right it worked I keyboard secret will be created so let's hit an apply let's go to key vaults and secrets and here is your secret which has your client ID and client Secret so let's see it shows secret value and this is the value so this is what we wanted to do all right now we are at the very last step so please bear with me for few more minutes you have shown a great courage till now so just for few more minutes and I promise it will really help you understand how to provision AKs using the right way so let's create a new module for AKs side or modules directory create a new folder name it AKs create a new file main.tf for AKs to save some time here and I have already explained you the process so I have copied the configuration file from the terraform documentation over here and I'll just review it with you quickly right so this is the data source location so the first one is the data source to get the latest Azure AKs version right and we specify include preview as fall so that we would only get the stable version it will be stored in this current so the First Resource is kubernetes cluster I'll give it the name location Resource Group name DNS prefix kubernetes version which will be referred from this data source current dot latest version then a node Resource Group will be created and these are the details of a default node pool a name size it will be in multiple zones so I have specified one two three and enable auto scaling is true max count is three mean count is one of type virtual machine scale set these are the node labels and these are the tags then I would be creating this Azure kubernetes cluster with the service principle right so I will have to specify client ID and client secret these values will be the same values that we have stored in key Vault so that later on we can either reference it from the key vault at the runtime or we can use the output variables of service principles that we have used in key vault over here in this example we are using the output values then you can specify your Linux profile so this will be the admin username which you can use to SSH into your kubernetes cluster and you can use this public SSH key it I'll show you where I have specified the SSH key and that these are some default settings right so we have used a variable of type file and we can specify the same way that we have specified the string variables so let me just copy the variable files as well so I'll create a new variable file in AKs call it variables.tf and these are all the values right so this is how we declare our SSH public key and in this particular location I have my Pub Key and you can basically create a new SSH keys and private Pub Key pair and specify the location over here right the same way we are using client ID client secret and location Resource Group service principle in the same way that we did for other resources I will not be explaining those again uh just to save you some time right and so yeah so this was the module for eks now let's go to our main.tf okay and use module name is AKs right first one is Source we will Source modules slash AKs and we can pass certain values from here like we did for other resources so I will be passing service principle name client ID client secret location and Resource Group name at the service principle client ID and client secret will be coming from the output variables of service principle so I'll do a TF in it again to install the AKs module once it is installed I can run TF plan okay guys I guess it is working now so let's do TF apply one for the approve okay now it says kubernetes cluster is being created it will take some time for it to provision but we can see in progress over here if we go to Resource groups and so there is this node Resource Group created for node pools and over here you could see the kubernetes cluster once it is created let's see if it has any resources till now oh yeah so it has an agent pool net for Security Group a virtual Network public IP address load balancer so all those all those resources have been created already and our kubernetes cluster will be visible there it is cluster is created and has all the details not pool configuration it took the latest kubernetes version for azure 1.24.6 and if you go inside the node pools you will see the details there is one default not pull which is still creating and it has one count but it will be we have specified node count as Min 1 and Max 3 so it'll Auto scale accordingly as per the workload and rest everything is what we have specified okay so it's still creating once it is created make sure to delete these resources from terraform itself else you will incur a huge cost for just the kubernetes cluster itself so make sure you do that thank you very much for watching this video I hope you have learned something out of it and it was helpful to you if you like the video please give it a thumbs up and subscribe the channel to get notified about all my upcoming videos and there will be lot more coming in the future so stay tuned have a good day and I'll see you very soon with the next video thank you very much for watching

Channel: Tech Tutorials with Piyush

Views: 20,285

Keywords: Deploy AKS Cluster in Azure With Terraform, how to create aks cluster in azure using terraform, azure service principal terraform, azure terraform, terraform tutorial, create aks cluster using terraform, deploy aks cluster using terraform, terraform tutorial for beginners, aks azure terraform, aks terraform tutorial, deploy aks using terraform, aks with terraform, devops project, devops realtime projects, azure realtime project, kubernetes, terraform, aks

Id: I-MbnfNcikk

Length: 38min 49sec (2329 seconds)

Published: Sun Oct 09 2022