Full Tutorial: Deploying Helm Charts in Kubernetes with Terraform

i have here the git repository that's also linked below so you can check that out with everything that we're gonna spin up [Music] hello there and welcome to another hands-on full 80 set tutorial here on my youtube channel for those who are new to my channel welcome my name is anaes and this is 100 days of kubernetes day 45 the challenge where we aim to learn something new related to cubanitas across 100 days now today we're going to be looking at terraform specifically how we can spin up a terraform repository that manages our infrastructure and the applications that are installed on our infrastructure now specifically we're going to first set up a blank new repository then we're going to set up our terraform providers we basically have to tell terraform about the type of resources that it has to learn about in order to spin them up and manage those once we have those initialized those terraform providers we can then spin up a blank new kubernetes cluster a siever cubernitus cluster and on that sivo cubanitas cluster we will want to install two applications first of all we're going to use a custom helm chart within our repository and then we're going to use the cute prometheus stack operator handshake from the internet i'm going to show you how you can manage those through terraform now for this tutorial you don't need any tarot from experience however it's useful if you already have some ham chart hum humming experience before i have about three videos or more on helm specifically where you can learn how you can get started with helm they are linked up there so check those out if you're completely new to helm now these tutorials they take me a long time to prepare since i'm spinning up the entire repository and showing everything from a to set also with a blog post you can check out if you prefer the written content the blog post link below so if you do enjoy this type of content if you enjoy this video if it's useful for you please please please make sure to hit the like button subscribe to my channel for upcoming videos and hit the bell notification icon i also have bi-weekly live streams so make sure to join those if you have any questions if you have any questions in the meantime we also have a discord community chat also linked below where you can drop in share resources related to the devops space to kubernetes to infrastructure management or similar also if you enjoy this type of content i have a weekly devops newsletter where share free online learning resources from across the space from amazing people such as yourself right to your inbox so please if you enjoy this type of content and you want to support me and my content please make sure to hit the like button and subscribe that would mean so much to me and let's get started okay so this is the tutorial on my notion page i also have it on my blog post link below so you can check it out there as you can see there's quite a bit of stuff that we're gonna do in this tutorial i'm gonna walk you through everything so that's why it might take a little bit longer to do now why do we want to use home charts to manage our applications a lot of people especially i've seen those comments on reddit suggest that this is an overkill you don't need home charts to manage your applications i disagree with that i would actually argue that even with simple applications you can use home charts to manage those and i'm going to show you in a second how you can use even simple harm charts to manage your applications the thing is with harm charts let me zoom in here you always have you have the templates you have your custom resource definitions so depending on the size of your application and the size of what you want to install basically you have different folders and they are installed chronolog chronologically meaning for example the codes are installed first before the templates and such the deployments are actually installed so if you have more complex applications there's a lot of logic that help manages for you there's versioning and how managers for you and a bunch of different benefits the main one is however if you use a harm chart such as this one you can modify the values so these are the values that you can pass into your home chart that you can modify depending on your needs so for example at sebum for our sre work we're gonna be using a lot of the helm charts such as this one and we modify it based on our needs we can modify the different variables that are basically specific to our infrastructure and it makes it a lot more easy to manage through home because we can see what is actually installed versus having to we'll put it directly into the yaml and manage it directly through the ammo we can manage it through hamchat so cube from if you're stuck that's the home chart they're obviously a bunch of different ones um however it's really easy to install those to get started they are managed by the community so you don't have to maintain all of the resources yourself in many cases there are lots and lots of benefits however installing helm shots one by one is obviously really tedious especially if they are dependent on each other so this is where terraform comes in now telephone has different providers such as the home provider and the home provider basically allows us to tell terraform hey there's this provided this resource this type of object that i want to use and that i want to tell you more information about on how you should manage for example those helm charts so you can have a look at this provider to see additional configuration how to set everything up and let's just get started i have here the git repository that's also linked below so you can check that out with everything that we're going to spin up now let's check that out here within my vs code so here's the readme with the instructions which are a bit shorter than what i have in a blog post so first we will have to set up our provider now this is the provider i'm basically providing the receiver provider and then also did here c for terraform provider and then i'm setting up the kubernetes provider and i'm setting up the helm provider now in some cases i might want to actually use cubanitas directly and not helm in in some rare cases but as you can see helm basically wraps the kubernetes provider in itself so we're going to set that up first you can take that directly either from the git repository or like if you clone it or you copy paste it um however this is what we're going to do first we're going to take a different and empty um repository that's over here so this one it is nothing so we're going to open it up first and then within that repository we're going to set up a provider.tf file t f there's the f and then we're gonna paste in our provided information now in this case let's leave out the helm provider or at least in the blog post i'm installing that later on you can do it right away or you can leave it out and follow the blog post now i'm setting up everything at once right now now let's open up a terminal and you want to have terraform installed now you can install it really easily through brew so i've listed some of the installation options here right at the beginning so you can check those out so you can basically brew install and then here through tap hashicorp tap and then you can install the terraform modules i guess that's how you call them so and then once you have that you have teraform installed and you have different options here for terraform right so we are not going to do much customization um what we do however need is here we need a variables file since we have here our receiver token they'll be passing in so what you can do basically you can set up within terraform you can set up a variables dot tf file and that variables.tf file allows you to specify variables dynamically so for example let's say i have a variable that i want to use a multiple parts within my repository right i don't want to hard coat them everywhere right i don't want to put everywhere i will show you an example in a second instead i want to define them in one place dynamically where you can change it in that one place so here's my variables.tf file as you can see i have here mysevo token that i just referenced and then the cluster name that i want to use the number of nodes and so on so i'm just going to use this and paste it into my variables.tf file okay so that it basically will have access to my civil token now i need my cvo token so if you don't have a ziva account yet then sign up to sign up to cvo here's my sifaccount this is my dashboard now here as you can see i have one cluster right now running within the london one region you can also use new york which you would have to modify within tara from itself i'm going to show you that in a second let's go to account go to settings and then security we have here receiver token and i'm gonna change that up after this video so don't even try to use it and i'm gonna pass it in okay so it can use it now i want to initialize the provider now it has access to this variable called sibo token here where deceiver token that's basically how you reference it so i'm going to say terraform and then init in it it's correctly referenced and then it's basically taking all those providers and taking and basically starting a dot terror from file with which references those providers i'm still learning about a lot of the stuff as you can see here's the hashicorp one with the helm provider and the kubernetes provide and here's my siever provider everything set up here now what do we want to do next we want to spin up i see a bookkeeper need this cluster so we're going to say cluster.tf and in this file we're going to specify what type of kubernetes cluster we actually want to set up right so let's go that's not what i want to open let's go back to the other file and here's my cluster resource now as you can see i'm also specifying by that cluster name where the applications and applications i want to install on my cluster the number of nodes that i want to use the region that's all specified within here so this is basically my um kubernetes cluster resource and then i'm also specifying the instance size separately as like a data resource and that depends on my civil kubernetes cluster this one depends uh basically that has to be installed first that like to get the data out of it it has to be installed first so this has to be spin up in order for me to get the data out of it that's basically how i understand it um so you have to in some cases you have to have a depends on because the thing is i want to obviously only move forward with my helm release installations once the kubernetes glasses spin up so if i put the depends on here then i'm making sure already that receiver cube needs cluster is already spun up before i'm actually moving on to the next steps so i'm going to take that and then i'm going to paste it in here and now i want to go ahead and i want to install this humanities cluster so let's do a terraform plan and the terraform plan will basically show me kind of like a dry run of what's gonna actually spin up what is terraform actually gonna manage so in this case i don't have to specify much more we're gonna dive in next tutorials in more detail and more advanced usage of terraform so in this case it's gonna say it's gonna spin up one new resource it's zero to change zero to destroy i didn't delete any resource i just wanna spin up a new resource so let's go ahead and also add an output.tf file and i'm going to show you in a second y dot tf5 so this is the output.tfm and in this file we can specify what i actually want to what what information i want to have displayed upon the visas being installed so for example i can say that i want to have access to the kubernetes cluster config that's going to be created right so i can specify then get access to the data that uh terraform now knows about of like my civil kubernetes cluster and the dot cluster resource that's basically how you specify it it's here the cuban need is cluster typically and then the name is cluster in this case that's really creative name and so let's go back to the output section so this is the different outputs that i want to have including my cube config to be able to connect to that cluster right so um i have now my cluster tf my output df my provided tf and my variables.tf i hope it's clear what all of those are i also have it in the git repository i have in the readme i have it in more detail defined um over here what are those files for that we're going to spin up so you can always cross check those also within the blog post so okay this is all set up now we can do a terraform apply right i wanted to apply my cluster resist i want to spin up our new cubanitas cluster now you can also set up the entire repository and didn't do everything at the end at once you don't have to do it step by step of first setting up the cluster right but in this case i just want to show how you can use terraform to just spin up a cluster itself and manage that cluster right you can for example then have the different resources defined so this is different to pressing buttons this is more like the github's way of defining resources of everything is going to be defining git right this is all all those files are committed in git so you can just have a look at what's actually running what kind of resources am i creating and that makes it easy for you to create exactly the same environment but also as a team it allows us to have reproducible workflows in the long term now there are a bunch of benefits you can check out my argo cd video if you have which includes a lot on git ups if you have further questions so let's apply this resource yeah let's go ahead and apply this resource so this is gonna create a tab from dot tf state file which is basically going to manage to state now it's going to tell me again what it's going to be creating so in this case it's just going to create a resource before you apply you always want to make sure it's actually creating the right resource right so in this case this is the right resource so we're going to go ahead and we're going to apply it and this is going to create my civil kubernetes cluster now this is going to take a few seconds right so in the meantime i'm going to show you what we're going to do next once we have that we're going to set up a charts directory and within the charts directory we want to have our custom charts now in this case i just used the default nginx chart that comes with it now you can modify this chart obviously as well we can modify this chart right um i can show you in a second also how to modify it but that's kind of beyond the scope of this video so we're going to set up a custom chart within the charts directory and once we have that we want to create a home release for that chart so this is basically our resource that we're going to create so type helm release it's an nginx chart and we just call it nginx um but here this chart name is important because it's going to look within the charts directory it's going to look for the nginx chart and then it's going to create a new namespace also called nginx in which it's going to install that child so i'm basically telling it what to install and where to install it to with this little file so let's see if this already spun up it's still creating it should take like about one to two minutes to create let's have a look at the dashboard and see it being created here so a tarot from example chart no this is just spinning up right now let's fast forward until it's done oh and it's done that was not very useful okay so going back as you can see here it applied the resource correctly and here's my outputs here's the cubeconfig that i want to use so let's copy this cubeconfig here from api version and let's just create a file called cubeconfig cube config and we're going to paste it in here our cubeconfig okay and once we have that we can say export and we can basically say our cube config is equal to this cube config over here right so once i say that see my cube i need is cluster here oh you can't really see it um let me change this up let me change this let me put myself over here for a second so you can see that the cube config here changed from my demo example cluster to my terraform example cluster now both of those are managed within civil so both of those are here name example and tara from example so now i'm connected to my terraform example cluster over here okay so now that we have that we can connect to it we can say cube cuddle somebody asked or told me that i shouldn't have an alias of like cube tunnel i do have it set up i just usually use cube color since if you don't have it set up it might be confusing if i just use k anyway so cube color get notes here let me show you what i just talked about okay cube color get notes so as you can see these are my three notes that are just gonna have been spun up and cube cuddle cube cuddle get namespace as you can see here my default namespaces now i don't have right now um the nginx namespace setup right so let's install that let's create a charts repository and then within charts cd charts what we want to do is we want to help create and create an nginx chart now we're just going to call this chat engine x you can install any other custom chart that you want to we just going to use the normal nginx chart right hand create nginx and then within here if we refresh we will have our nginx chart now this is the normal template chart with the helm create command that comes automatically okay so once you have a hang chart set up independent of terraform in the pin of any resource that you use what you want to use is to test your home charts let me just focus on the terminal for now you want to use the tree nudity allows you to easily test your kubernetes manifests whether those are pure yama manifests or those are health charts let's have a look at the tree website the dream getting started now this is the docs we can also go directly to the tree now the goal is to prevent kubernetes misconfigurations from reaching your production clusters right before you install anything you want to make sure it's correctly set up now especially when you're getting started with kubernetes it's quite difficult to know what actually has to be included within your manifests and so the tree allows you to set up policies basically what has to be true or how do you manifest have to be defined in order to pass the tests right so you can use the tweet directly just by getting started through the simple curl command just here all the different options on how you can get started it also has a docker image so you can use it also in uc icd pipeline there are also different options of using a new ci cd pipeline now then when you run did you test and you specified a file so let's go back to our terminal d3 test it's gonna test like this example file in this case now in this case for this example file it's correct set up everything passes now let's check if our example chart that's directly pulled from helm passes as well so for that we will need the day tree hum plugin here the treehound plugin now you can directly install it with home plug-in install you can extend home with different plug-ins it's really really useful it's really handy so in this case i already have it set up the plugin is already installed so i can directly use it with home day to test and then my charts directory i can also pass on the specific values that i want to use or similar like how i would use helm but in this case i'm going to use the date free test command from the plugin so i'm going to say going back going back over here helm the tree test and then this is from the root directory now in this case i'm already one directory up so we can just say help day to test and then nginx okay we want to test that chart so now it's directly testing our home chart and as you can see six of the 19 rules did not pass now you can have a look at the specific rules but just by following this link that's provided to you once you use the dream where you can see the different policies that that are basically set up you can modify them but the goal is ultimately to modify your helm child right the goal is that you modify the sound chart to pass now this is for another video i'm not gonna go ahead and modify that right now i'm gonna go ahead and actually trust the helmentainus for creating this chart in a okay manner so now we have the snapchat we want to manage it through her terraform right so we say hum release and then we say nginx dot tf right and in this file we're going to post the home release for this chart right so this one what i told you about earlier so we're just going to post it in here and then in addition to that we also want to use the helm release engine and prometheus prometheus.tf okay that's our other home release that we want to set up so going back i'm just going to copy paste that it's a similar process over here so you have to name in this case we're just going to call it prom and the child is called cube prometheus stack that's the chart but it's going to install all of our prometheus resources now this is the repository link you can find that directly within the git repository i'm going to create a new namespace called monitoring in here create namespace 2 since it doesn't exist yet and then i'm going to basically install that help chart over there now you could set different variables you could say set and then you could basically say key value i think something like that so you can pass in the different values into the hum chart so you don't have to have a value cml file such as here you could for example say image dot repository that's the value and or like the key and then the value is nginx in this case so you could pass it in directly over here you can show that in more detail if you're curious and how to do that so you might wonder how does terraform know about which cluster these resources to install to this these two to install to well we have here a cubeconfig so we're connected to it right now but that doesn't matter to ter to terraform terraform doesn't know about the cluster that i'm connected to because i'm telling it in the provider that it has to through the helm provider it has to connect to the data.cevocubernet.cluster.cluster object like this object that's stored within my data resource i didn't i hope i used the right terminology so we can say terraform and then state list and this will give me the states that terraform is tracking right now so i can for example take this resource here this data.civo uh kubernetes cluster dot cluster and then i can go ahead and say terraform console and within the tear from console i can query that object that it knows about so once the console starts okay cool and once i query it i get all the information on this cluster right so i know what's within the cluster what i can use to actually connect to it that's basically the information on the cluster that we created earlier right and i just did that you can create everything all at once i just did it to show you how i um exit so um how i tell tara from where to install those helm charts too right so let's do another terraform plan to see whether those two hand charts will now be created right within our cluster if it can if it can actually create it so let's do a tara from plan and wait for it to process so as you can see here there are two more resources to add so to change you to destroy now which are the resources so first is a home release called prometheus in this case and the second one is the hum release nginx right these are the two resources that we want to create so once we're happy with it we can go ahead and do terraform and then apply of those resources right we want to apply them to our cluster now this will then create a state and make sure that the um basically it compares our actual state within our cuban needs cluster with our desired state that we have defined here so this is our desired state these resources define what state do we want our resource to be in what state do we want our cluster to be in we want to have those two hamsters installed now that means if i can't find those two home charts being installed and managed through terraform in the cluster that it will have to create them so in this case it's basically just telling me these are the two resources like we just looked at that it's going to create and i can say yes i want to create them on my cluster right and now it's gonna make sure that it's gonna install this resources so the actual state is the desired state of our cluster right so those are both the same now as you can see it's gonna create our home release nginx and our home release prometheus it's gonna create both at the same time because they are not dependent on each other now what do you do if you already have hamstrings installed within your cluster you will have to pass in additional labels or annotations on your um like on those hem charts so terraform can actually take over the management of those hand shots that are already installed within our cluster the easiest way is obviously to do directly through terraform versus first installing helm charts and then making sure later on the diminished with terraform right so let's wait until this is all finished and then we can take a look into our cubanita's cluster and make sure that everything is running correctly and set up correctly and the chart and everything is working now at this point i want to give a special shout-out to the tree who helped me in supporting this content and collaborating with me um to make this possible to make these kind of tutorials possible because it obviously take a long time to create so if you enjoy this kind of tutorial and you want to make sure others see it too please hit the like button and subscribe to my channel for upcoming videos that would mean a lot to me also i have a weekly devops newsletter where i share free online learning resources from across the space to your inbox from amazing people such as yourself so make sure to subscribe to that but yeah it would mean a lot to me if you can hit the like button on this video subscribe to my channel for upcoming videos and let's continue with the video so as you can see this is again the output since we specified that this is always going to be the output that's going to be provided to us when we spin up resources uh it's always going to provide those it's not just going to provide us right now when we create a cluster so as you can see those two pieces have been created so we already connected to that cluster through this cube conflict that we specified earlier so we're just going to say cube cuddle get namespace right we want to see our namespace and now we can see that the nginx and the monitoring namespace have been created so kubecon get all in namespace nginx and as you can see here this is everything is created now we can connect to this service right we can do a port forwarding on the service to make sure it's working so uh cube column and then we say port forward and off the service and the service is in namespace engine x and we want to have let's say 9000 going to 80 since this is running on port 80. it shouldn't run on 480 but it is on port 80. so let's see if we can open it up on localhost 9000 localhost 9000 and it says welcome to anginax now you can obviously modify it you don't have to run nginx uh in this case i'm just using the default child right to show you how you could use custom charts within your terraform repository now this is working let's also have a look at um the other one cube color get all in namespace monitoring and as you can see here is our entire deployment from my cute prometheus stack operator chart now i want to connect to this one this service the cube prometheus stack prometheus service so i'm going to do the same but with this port forwarding so port forwarding and we're gonna do 1990 2.19 90 to put 1990. it's a namespace monitoring this case monitoring and we're going to change up the service since it's a different service and let's port forward that and check out 1990 that we can actually see prometheus as you can see everything is working it's managed right now through terraform it's installed on our cluster now you can spin up everything at once you don't have to do it in this chronological order like i did if you prefer the written content again blog post link below now this is it for today i hope this video tutorial was useful if it was please remember to hit the like button subscribe to my channel and if you prefer the written content check out the git repository link below as well as the blog post if you have any feedback any comments suggestions anything you would like to see in upcoming videos please also comment those below or drop into our community chat and just start a conversation there i hope to see you in upcoming videos have a lovely day bye you

Channel: Anais Urlichs

Views: 2,970

Keywords: kubernetes, gitops, devops, kubernetes tutorial, what is kubernetes, kubernetes tutorial for beginners, terraform, infrastructure management with terraform, kubernetes infrastructure, terraform for beginners, Kubernetes testing, terraform tutorial, terraform tutorial for beginners, Helm Charts, Managing Helm Charts with Terraform, Helm Chart Deployment, infrastructure as code, infrastructure as code terraform, GitOps best practices, gitops kubernetes, infrastructure as code aws

Id: Qq1cfVw1Mx4

Length: 32min 31sec (1951 seconds)

Published: Wed Sep 29 2021

Reddit Comments

[u/kamikazechaser]

Some consider this as an anti pattern. I personally moved away from terraform for helm charts to Helmsman. But terraform works out for some people.

[u/vsimon]

If you're looking to declaratively install and manage Helm charts, check out Helmfile

https://github.com/roboll/helmfile

[u/mahty_mcfly]

Nice! I’ve used this method for deploying Helm charts that I consider infrastructure, and don’t change often enough to warrant an a full phat CI/CD pipeline (I’m thinking Prometheus off the top of my head). It’s useful if you already use Terraform for deploying the rest of your cloud infrastructure.

[u/milkcurrent]

Terraform's Helm support and general support for Kubernetes is a garbage fire. Use literally anything else. We also use Helmfile after violent rejection trying to deploy charts with Terraform.

[u/seeker_78]

Certainly useful when you want to inject terraform created Infra component resources like IRSA that are created as part of bootstrapping cluster cluster component.. needless to say a that you don't want to expose these tf resources, rather inject them on the fly

Any use case beyond this is a stretch and I agree it is better managed with helm or CD pipeline