Terraform with Azure Kubernetes Service

[Music] what is up youtube and welcome to another video in a previous video we've taken a look at how to get a kubernetes cluster running in microsoft azure if you haven't seen that video take a look at the link down below i take you through all the steps of everything you need to know of how to get a kubernetes cluster up and running in azure today we're going to be taking a look at the same thing but how to do it all in terraform so without further ado let's go so the first thing you're going to need is an azure account so what you want to do go over to google type free azure account just scroll down look for the create your free account today link once you're in that link this will give you access to 12 months of free services and everything you're going to need for this video if you already have an azure subscription in the next step i'm going to show you how to interact with that subscription [Music] now if we head over to the terraform documentation you can see they have an azure provider and this is basically a module that allows us to interact with microsoft azure and spin up our infrastructure in the cloud now terraform is great because rather than using bash groups or powershell scripts and creating our infrastructure line by line or script by script we can just create a terraform file and let terraform do all the work and that becomes our source of truth and our infrastructure as code now to do that we're going to need to authenticate with azure there's a couple of ways to do that if you're running locally and you're the one running terraform all the time and interacting with terraform they recommend to use the azure cli so it'll just use your credential but if you're running it in a ci cd manager like in jenkins or somewhere in the cloud you're going to want to use a managed identity or service principle to do the work for you so in this video i'm going to be using a service principle and i'm going to give it the permissions it needs to interact and manage our infrastructure so if you're new to this channel everything i do on this channel is on github if you clone the source code you see the docker development youtube series and if you go down we have a kubernetes folder and we have a cloud folder so all my kubernetes in the cloud series is all in here amazon azure digitalocean google and lenode today we're going to be looking at azure and for this example i've created a terraform folder with the readme inside it so everything i do is documented in here so you can follow along so the first thing we're going to need to do is get the azure cli so if we go over to docker hub and we search for the azure cli we can see we have the the cli tools here to interact with microsoft azure so the first thing we're going to need to do is gather some information so the steps we can follow here is to run the azure cli in a docker container this is the easiest way to get the cli up and running without having to install the dependencies so what we do is we say docker run interactively we mount the entire git repo into the container into a folder called slash work and then we set an entry point to run some bash and we're just going to run 2.6.0 of the azure cli then what we're going to need to do is gather some information about our subscription that we can pass to terraform so to do that we're going to need to run az login so what you're going to want to do now is copy this url and open it into the browser and then just follow the prompts you're also going to need to paste this code into the browser when it prompts you and then give it your account details now once you've logged in you'll see that there is a tenant id you're going to want to grab that entire tenant id and create an environment variable called tenant underscore id that allows us to use the the parameters much easier so then the next thing we're going to want to do is list out the subscriptions that we have access to so i'm going to say az account list and i'm going to list out our subscription because we're going to need the subscription id and to grab that you can see here i'm also going to create an environment variable and i'm going to grab the subscription id and i'm going to pass it into the environment variable called subscription and then just to make sure that any commands i run runs against the subscription i can run the az account set command and pass the subscription id that'll make sure that every command i run will run against that subscription so i don't accidentally do anything to my production subscription now in order to interact with azure using terraform or the cli we're gonna need a service principle a service principle is just a service account that we can use to interact with our azure infrastructure so all the steps you're going to need is all documented in here so the first thing we're going to do is create the service principle using the az adsb create for our back command we're going to give it a name called ak is getting started dash sp and i'm gonna output the result of the creation as json so to view that service principle information i'm just gonna echo out that environment variable and we're gonna need two pieces of information here the app id which is the service principle id and the secret which is the password over here so we need to keep the app id and password for later use the next command we're going to need to run is we're going to create a separate environment variable to hold the app id i'm going to say service principle equals and i'm just going to echo out that json and grab the app id that we have at the top here i'm then going to do the same for the secret i'm going to say service principle secret equals and i'm just going to echo out the json and grab the password field over here now currently the service principle that we created has no permission so we need to give it contributor permissions to the infrastructure that we want terraform to manage so to do that it's very simple i'm going to run the az role assignment create command i'm going to pass the service principle id and i'm going to give it scope over my subscription i'm going to give it contributor rights this will allow terraform to manage infrastructure on this subscription and service principle also allows you to give more granular details so you can give contributor over a specific resource group or whatever infrastructure you want to reform to manage so now that we have a service principle the next thing we need to do is go and grab the terraform cli so to do that i'm just going to run curl and i'm going to grab the terraform 0.12.28 binary so i'm going to go ahead and download that and then i'm going to also run the unzip command to extract it so i'm going to unzip that and i'm going to give it execution rights and i'm going to move it to user bin that means i can now have access to the terraform command so what i'm going to do is i'm going to change path and i'm going to change path into the kubernetes cloud azure terraform folder and you can see now we have an an empty folder here there's nothing in here so how do we start spinning up our aks cluster in azure the first thing we're going to need to do is write a terraform file and we're going to have to import the azure rm module and then authenticate with azure so the first thing i'm going to do is i'm going to create a main.tf file in this terraform folder so if you do ls you can see i just created a main.tf and the other thing i'm also going to do is create a variables.tf so this will allow me to pass information to my terraform file so what i'm going to do is i'm going to install the azure provider so i'm going to say provider azure rm i'm going to import version 2.5.0 and i'm going to pass a bunch of things to authenticate with azure using terraform so you can see i have a variable called subscription id telling us what subscription we want to access i have a service principle id and key that's the service principle account we just created and an azure tenant id now if i go over to the variable section we have to actually declare these variables here so that's very simple we just use the variable block and we give it names so you can see the names i have over here service principle id key 10 and id and subscription id and they match the ones over here but that's how you pass information from terraform into your main file now to test whether this works we're going to need to run the terraform init command and what this will do is initialize our file it'll download any modules and providers that we refer to so you can see here it says checking provider plugins it's going to go ahead and download the azure rm plugin for 2.5.0 and it will initialize terraform so we're pretty much good to go it's also going to store this all of this initialization files in a dot terraform folder you can see it's downloaded all the plugins over here so one of the cool things about terraform is it has a plan function which basically shows you the execution plan of what terraform is about to do so you can run this command when you've made any changes and terraform will tell you exactly what updates it's going to apply if any so what we're going to do is we're going to test this out by running terraform plan and then we use the dash var argument to pass in environment variables now we have to pass in all those variables we've defined and this helps us keep secret information out of git so we can say service principal id pass that in we can say service principal key pass that in 10 an id and the subscription so if i go ahead and run this we can see that everything is all good terraform keeps an in-memory state of the plan and it keeps this inside of a terraform state file so you can see here there's no changes infrastructure is up to date because there's no actions that we gave it to perform now terraform has the concept of modules modules are a great way for stitching together multiple terraform files which keeps your main terraform files simple and easy to read and understand so what we want to do is we want to create modules we don't want to literally put everything into the same file because it'll make it really hard to read and digest so in my terraform folder i'm going to create a new folder called modules and i'm going to create a separate module for spinning up azure kubernetes i'm going to create a new folder inside here called cluster and you can create any kind of module folder structure that you want that makes sense to you so i'm going to create a new module called cluster and inside here i'm going to create a cluster.tf file so this will represent our azure kubernetes infrastructure now for this module we're also going to need to specify some variables so i'm going to create a variables.tf file because we're going to want to pass variables from our main file into our child module now because azure kubernetes is going to need a service principle in order to spin up things like load balancers and storage and things like that we're going to need to create our first variable in the cluster tf folder um in the file called service principle id and service principle key because we know aks needs a service principle in order to function so now we can start populating our cluster file now we're going to need two things in the cluster file the first thing we're going to need is a resource group we need to create a resource group to put our azure kubernetes service inside so to do that i'm just going to create a new resource inside of our cluster tf file called azure rm resource group and i'm going to give it a name called ak is getting started so the name of my resource group is going to be aks getting started and we're going to deploy this to a location from the variable from the our variable called location so we need to define that variable so if we go over to the variables file we need to specify a location variable so i'm gonna just put a location here and i'm gonna get a default value of australia east i've referred to the azure rm resource group resource now there's a bunch of resources if you go over to the azure provider on the hashicorp that documentation and you scroll down there's all these different resources you can apply these literally represent all the resources available in microsoft azure so today specifically we're going to look at container resources and we're going to take a look at the azure rm kubernetes cluster resource this has everything to do with how to provision an aks cluster using terraform so now if we go back to our cluster tf file we can see now we've told azure to create a resource group so the next thing we're going to need to do is we're going to need to to tell it we want to create a kubernetes cluster so to do that i'm going to create another resource in here it's going to be the azure rm kubernetes cluster and i'm going to call it aks getting started i'm going to give my cluster a name and i'm going to give it a location as well and i'm going to use the location of the resource group so you can see terraform gives us the ability to stitch up variables and between different resources so here i'm saying i want to use the resource group called aks getting started and i want to use its location and its name i also specify the resource group i want the cluster to go in and then i give it a dns name and i also provide kubernetes version from a variable so we're going to have to go back to our variables file for this cluster and we need to pass in a new variable for the kubernetes version and i'm just going to run 1.16.10. now with azure kubernetes we have to provide a default node pool so to do that inside of this resource i'm going to add a default node tool block i'm just going to give it a name called default and we're going to have one node this is the vm size we're going to use and we're going to use vm scale sets as the type for this node pool and i'm also going to say i want 250 gigabytes of this all the fields that you see here are basically the same fields i used in my introduction to aks video guide using the azure cli so terraform has its own module where we can pipe in all these different fields and we can basically do things like create multiple node pools we can provide v-net ids if we want to integrate our cluster into an existing v-net and a bunch of other cool things so the next thing i'm going to need to do is provide a service principle that that aks can use in order to function so i'm just going to create a service principle block and i'm going to create a client id and a client secret and i'm going to pass in the variables that we have defined already service principal id service principal key and if we go to the variables we can see those two are over there ready to go the next bit i'm going to need to do is provide a linux profile because our default node pool is going to be linux i need to provide a username and an ssh key for this these machines so that we can use ssh to access them in case we want to so to do that i pass in the linux profile i'm going to give it an admin username called azure user and i'm going to give it an ssh key again from an environment variable so we're going to go through and go back to the variables and we need to define and a variable for our ssh key i'm going to just paste that over there then i also want to create a network profile so below the linux profile i'm going to create a network profile and this will tell um aks what network plugins to use i'm just going to use the default cubenet and i'm going to use a standard load balancer now the next block i'm going to add is just a bunch of add-ons in the add-on profile that i want to disable the aci connector i want to disable that i want to disable azure policy disable all these features such as the dashboard and the oms agent now if i go back to the terraform documentation you can see that they have azure rm kubernetes cluster this that is the resource block we've just defined and if you go through here they have a ton of information they have the add-on profile that i um use dns prefix default node pool all the stuff i've just shown you but there are a lot of extra things here if you want to play around i'm going to leave this document up to you you can add auto scalers and you can do a whole bunch of things like drop your cluster into an existing v-net and a bunch of other cool things so now that our terraform files are ready and our modules are ready we missed out one crucial part and that's the ssh key you can either bring your own ssh key or you can generate one like this so i'm just going to say ssh keygen i'm going to pass in a very super strong secret email address and i'm going to create that in that directory and then i'm going to cut out and pass the public key into an environment variable called ssh key this will give us the ssh key that we're going to pass through to terraform terraform will then pass that to aks and we can use that to access our machines running on kubernetes if we need to troubleshoot stuff so now if we go to our cluster module and we go to the variables we've added a bunch of extra variables now we've added location kubernetes version and ssh key we need to make sure that we define these variables on the outer file on our main terraform files variables as well so what i'm going to do is i'm going to go ahead and paste the extra information bits that we need the ssh key the location of the kubernetes cluster and the kubernetes version because what we need to do in our main variables we need to be able to pass it through to our child module now before we run the terraform apply command we want to see what terraform is going to do when we run terraform apply so we use the terraform plan command and we pass in all the variables i've just mentioned now when we run this we see that there's no changes so why is that that is because if we go back to our main terraform file all we're doing is just pulling in the provider template we need to actually go and execute our child module which we haven't done yet so to do that we need to create a module and block in this main terraform file so i call it i add a module called cluster that cluster represents the module over here called cluster and we pull in its source so you can see i'm pulling it from the modules folder from the cluster folder so terraform will know where to find that cluster.ta file i then pass in the variables for that module so you know we've defined in the variables over here we've defined service principle id key ssh key location and kubernetes version we then also defined it on the outer template in this main tf variables over here so we go ahead and pass that down to that module now to make sure that the uh terraform has is able to pull in our new modules we have to say terraform inlet and that's going to go ahead and initialize all the plug-ins and modules and things it needs and then we can go ahead and run the terraform plan command to see what it's going to do so now it says terraform will perform the following actions you can see that it's going to create our resource group it's going to create our cluster and down below it's going to give us a summary so it's going to say two plans to add nothing to change nothing to destroy so basically the two resources that it's planning to add is going to be in the cluster tf is basically going to be these two resources the resource group as well as the aks cluster but now that we know what terraform is going to do we can go ahead and run the terraform apply command with the same variables that we passed in before and if we do that terraform will tell us again that it's planning to add two resources and it's gonna ask us for confirmation i'm just gonna say yes and now it's gonna go over and start creating our resource group and creating our kubernetes cluster so this will take a couple of minutes to come back right now you can see terraform has gone ahead and applied the resources so it's complete if we take a look at the azure portal we can go to our resource group and we see we have a cluster up and running called ak is getting started it's been created it has one um node pool and running kubernetes 1.16 now one of the strengths about terraform is when we want to make changes to our kubernetes cluster terraform is really good now normally what happens in traditional infrastructure is people run powershell or bash scripts against their cluster and they don't have a source of truth of the state of the kubernetes cluster or the infrastructure running in the cloud and this is one of the strengths of terraforms the terraform files and modules become your source of truth if you want to make changes you can change the files and just reapply so what if we wanted to make changes to the cluster now one cool thing i'm going to show you here is that we can go to our cluster resource we see we have our kubernetes cluster here we can add another resource here called azure rm kubernetes cluster node pool so let's say we want to monitor our cluster we want a separate node pool for running all our monitor stuff so i'm going to create a node pool called monitoring i'm going to give it a name i'm going to point it to the same cluster id of the cluster we created up here so i'm going to say azure rm kubernetes cluster aks getting started id pass that idm and this time i'm going to create a different vm size i'm going to add one node count with 250 gigs of space and i'm going to provide an os type of linux and if we now run terraform plan command we'll be able to see what terraform is going to do terraform will now look at the cluster state look at its state file and determine the changes so you can see here it came back and said one plan to add nothing to change or destroy so terraform knows to go and add that node pool i can make a couple of extra changes like i can change some of the parameters um over here let's say let's increase the count of our default node pool to two so let's scale up our cluster as well as add a node pool to it so to apply these changes um rather than running powershell or bash scripts i can now just make the changes to the file and say terraform apply so now terraform will detect the changes and it will go ahead and ask us confirmation if it can apply the files and we're gonna just say yes and it's going to go ahead and apply it so now while terraform is busy let's go ahead and look and see what it's doing so we go to our resource group go to our kubernetes aks cluster we can see now we have two node pools so the operations are succeeding we have a default um node pool being scaled to two so we increased it to two that's one thing we asked terraform to do the other thing is create our monitoring node pool so it's going ahead and it's creating that node tool now for those of you who have worked on kubernetes before you would know that creating a cluster is not where it stops we normally have to go and provision all the infrastructure on top of our kubernetes that we need to make our kubernetes cluster operational these things can be ingress controllers they can be monitoring things like prometheus that can be logging services like loki grafana and fluentd now instead of having to go and access our cluster and then applying a bunch of yaml files to the cluster terraform has a kubernetes provider so it's able to do all of this stuff for us as well this is one of the really neat selling points of using terraform to spin up infrastructure like kubernetes so in order to do that now i'm going to show you how to spin up a deployment with two pods and a load balancer now in order to do that we're going to create another module so in this modules folder i'm going to create a new module called k8s and i'm going to create a new terraform file called kts.tf now you can create modules for monitoring ingress controllers logging services service meshes whatever you want to apply i'm just going to show you the fundamentals and then you can take it from there so in here i'm going to create a new terraform file and i'm going to use the kubernetes provider now if we go to the terraform documentation the kubernetes provider is listed here and it and it has a bunch of examples you can follow but you can see we can do a bunch of things like config maps demonstrate deployment pods services ingresses so all the yaml files that you have you can convert them to terraform files and the net terraform do all the work for you so what we're going to do is we're going to create our first file and we're going to call it the kubernetes provider and this needs authentication detail so how do we want terraform to authenticate with kubernetes so there's a few ways to do that but the easiest way is to tell it not to load a config file and we're going to pass the host of the api server of aks we're going to give it the client certificate and the client key and the cluster ca certificate of aks now as we know we get we're getting these from variables so what i'm going to do is i'm going to create a new um file in this kids folder and i'm going to call it variables.tf and then in that file we're going to have to paste those variables now these are the variables that terraform will use to access the aks cluster and deploy our deployment with pods and services so we defined that over there and then let's build out what we want to deploy so i've just i'm grabbed an example of a deployment so this is what a deployment looks like so we pass in a resource called kubernetes deployment and i'm going to call it example and then this is pretty much the same thing as the similar to the json and the yaml that you would pass to cube ctrl apply or your helm chart but here we have us a deployment spec so i'm going to create three replicas i'm going to create some labels and selectors and then i'm going to pass in the image i want to deploy so i'm just going to run nginx as this example i'm going to give it some resource limits i'm going to give it a liveliness probe so that is what a basic kubernetes deployment would look like in terraform now what i also want to do is i want to expose this deployment through a kubernetes server so i'm going to create another resource here called kubernetes service i'm going to call it example and i'm going to name it terraform example over here i'm just going to select my deployment that i created above and i'm going to expose port 80 and i'm going to do so via a load balancer now once we've created this module and we have to go and stitch it up to our main.tf file if we go over to our main terraform file at the moment we only have azure authentication and we have our kubernetes cluster that we want to create we now also have to tell terraform that we want to deploy some stuff to that aks cluster so i'm going to add another module reference here to our main file and i'm going to call it k8s and it's going to source it from this modules directory from this kds folder we just created and i'm going to pass it the variables that we need and this is one of the key strengths of terraform is that we can stitch up these things using variables and using variable outputs so take a closer look over here we have host client certificate client ian and cluster ca certificates these are variables that we defined in our k8s module now in order to get that we need to grab those from output of our cluster module now we haven't defined those outputs yet so part of the azure providers aks module it has a bunch of output variables so if we take a look at the terraform documentation we have the azure rm kubernetes cluster if we scroll all the way down we will see the attributes reference page of showing us what attributes are exported so whenever terraform creates resources it has the capability to export output so what we can do is we can actually output the cube config file um as part of that module so to do that we need to go to our cluster folder and we need to create a new file called outputs.tf so this output file will define whatever variables we want to output as part of the cluster creation so the first thing we're going to do is create an output block called cube config and we're going to output the value of the azure rm kubernetes cluster this is our cluster that we created and we're going to output the cube config raw that is the same value that's been defined in the documentation over here and then we're going to use this value in a couple of more output variables you can see i'm referring to them here by saying azure rm kubernetes cluster ak is getting started cubeconfig 0 and i'm pulling out the cluster ca certificate the client certificate the client key and the host and i'm pulling those out as different output variables and now we can go ahead and use these output variables from our cluster module and we can input them into our k8s module this will allow our kubernetes module to authenticate with the azure aks cluster and deploy our deployment to pods and a server now because i've added new modules i have to re-run the terraform init command so this will go ahead and download now the kubernetes plug-in as well and then to see what changes we're about to make we can run the terraform plan command and now we can see that there are two plans to add so those plans are basically defined in the khs tf file we have our kubernetes deployment which is one plan and if we scroll down we have a kubernetes service which is the other plan so terraform exactly knows what to go and change in our infrastructure now to make things a little bit more interesting what i'm going to do is i'm going to go to my cluster tf and i'm going to remove this node pool so let's say i decided i no longer need this node pool or monitoring i'm going to go ahead and delete that i'm also going to scroll up to my default node pool and say i no longer need two instances i'm going to scale it back down to one and then i'm gonna re-run the terraform plan command and see what it tells me now now this shows you the power of terraform so now we can see we have two plans to add that's our deployment and our kubernetes service and then we have one plan to change that's our default node pull on our aks that we're planning to scale down and we have another plan to destroy which is our monitoring node pool now to apply that i'm just going to say terraform apply and that's going to take a couple of minutes to execute so now that that's done let's see what we've deployed so i'm going to go ahead and grab my kubernetes config saying az aks get credentials going to pass in the name of my cluster and the resource group that's going to go ahead and grab the config downloaded and ready for cube ctl so what i want to do now is go ahead and download cube ctl just using the curl command and i'm just going to give it execution rights and i'm going to move it to user local bin and now let's take a look at what we've deployed i can say qctl getservice and we have a service exposed via a load balancer over this ip and i also have a deployment called terraform example with three nginx pods busy creating and if we go over to that ip address in the browser we can see we've deployed our nginx server so this showcases how easy it is to get infrastructure up and running and manage your aks cluster and all your fundamental services that run on top of that using terraform so hopefully this video helps you guys get started with terraform and aks let me know down in the comments how you guys manage your infrastructure and especially around kubernetes and also let me know down below what sort of videos you'd like me to cover in the future and as always like and subscribe and until next time peace [Music] you

Channel: That DevOps Guy

Views: 9,155

Rating: 4.9312716 out of 5

Keywords: terraform, devops, infrastructure, as, code, azure, aks, kubernetes, k8s, cloud, training, course, cloudnative, az, github, development, deployment, containers, docker

Id: bHjS4xqwc9A

Length: 27min 55sec (1675 seconds)

Published: Fri Jul 31 2020