Automate Homelab Deployment With Terraform & Proxmox

hey everybody and welcome back to Jim's Garage in this video we're going to be looking at terraform and before you worry it's not as terrifying as you might think initially when I first went down terraform I thought it was going to be a real struggle to get all of this working but thankfully it's just a few configuration files so why might we want terraform well if you've been following my videos around kubernetes and Docker swarm it can be a pain to set up your infrastructure one by one even with things like a cloud in it so terraform basically takes that principle of having something like a cloud in it or it could even be a standard virtual machine and it automates that process so by the end of this video you'll be able to run a single command a terraform apply and you'll have all of the virtual machines set up that you need for your cluster or for your home lab environment now terraform is a great thing to learn because in many large organizations it's used for large scale deployments whether on Prem and even in the cloud and it's similar to anible but not the same terraform typically is for infrastructure anible is typically for the applications that will run on that infrastructure and anible playbooks are great for making sure that states are met we'll come on to answerable later down the path but this video we're going to focus on getting the infrastructure done first so taking a quick view of the documentation before we get started we asked the quite obvious question of what is terraform well terraform is infrastructure as code so much like we have things like uh Dock and compose and kubernetes manifest files Etc we can do the same thing for our infrastructure in this case we're going to be spinning up proxmox virtual machines using clouded it and it means that when we run our terraform script yep there's a script with a CLI and I'll come onto the installation in a moment it means that that process is repeatable so whenever we run that script we get the same output I.E we get the virtual machines configured in the number that we want in order to spin up our cluster so how does all this work well typically and as you can see on here there's Integrations for many Cloud providers but what we need is a plug-in so typically these plugins and providers are what give terraform its flexibility so a plug-in is a bit like having a custom language that terraform can interpret and can be used to do things so we're going to be using one specific to proxmox and yes there are more than one that you can use for proxmox and there's a whole host of others depending on the application or the process you're trying to fulfill so whilst this is going to demonstrate using proxmox and it's not quite perfect there's a couple of bugs in it I want to step back from that and show you the overall process so that whilst I will be providing the config files for this hopefully you'll be able to follow the process and in the event that libraries or providers become deprecated you should understand what's going on and be able to switch your existing setup to use those instead so I'm now going to jump into a review of the script that I've put together and then we'll go through all of the steps that you need to actually get this to run we need to hop into proxmox and do a little bit of configuration we need to install terraform itself and amend a few values and you can decide whether you want to use something like Windows subsystem for Linux to deploy this or whether you want a bespoke VM regardless of what you choose I'm going to be doing this within vs code and using the SSH connection that I've shown previously to do all of this so the provider we're going to be using for this video is the tailmate proxmox now I had quite a bit of fun whilst I was researching this video fun in so much as I was pulling my hair out and it didn't work but thankfully if we hop into this tab on that GitHub you can see that as of 2 days ago there's been a resuscitation of this project and it's now almost working it's working enough that I can show you how to deploy it and I'm hopeful that over time this is going to get better there is a competing one by bgp if you want to use that I haven't used it because this did what I needed to and again the point of this video is to show you how this all works and the processes so that you can replicate and switch between providers and hey you don't even have to use this just for proxmox there's a whole poost of other applications you could use this for but anyway handily with this provider as you can see on screen it comes with some examples and in here we get examples for cloud init which I based my example from albeit this one doesn't work exactly as it should that's because as I said the provider has become deprecated over time because the proxmox API has changed with newer versions and that's something that's probably going to evolve over time there will be changes to the API and unless the the provider is changed as well things just will not work as expected also there's things for lxc PVS which are just your standard virtual machines and also vagrant I'm only going to be doing the Cloud in it for this one because I think it's the most sensible Choice especially when we're wanting to do things at scale we get all the benefits of cloud in it I.E a preconfigured virtual machine with all of the settings already applied go and check my video If you haven't and then we get automation to replicate it yes I know you only have to right click and click clone it's a pretty simple process but if you're doing this at Enterprise scale it would save you so much time so I've just hopped over onto vs code and by now you should see in the bottom that I'm sshed into a virtual machine now this virtual machine I've just spun up it's an auntu server and I'm going to use this to deploy the terraform script you can use whatever you want you could even use that windows subsystem for Linux the first thing we're going to going to do you might have guessed by now is we're going to head over to our friendly extensions Tab and we're going to search for terraform this is going to allow us to have nicely formatted code and it should be able to do things like Auto indentation lint cleanup all of that good stuff so now that's installed we can head over back to our files and I'm going to copy over the files that I've created for this video so by default we're in the home directory here and I'm just going to copy these files over now before we talk about them so the two files I'm going to use in this example I've called providers and test cloud in it now they both have the TF file extension for terraform now there's not technically any right or wrong way of doing this I've simply created two files here but you may create more say for example you wanted a TF file specifically for your secrets you can create variable names and then reference them a bit like you would do Docker Secrets perhaps in a separate EMV file where you can lock down some of the permissions so let's have a quick look so in the providers file you can see here I've created a provider I specified a version and then I created a separate provider for something else so the first one which is in these terraform parenthesis you can see that I've created a registry and a specified aversion so this process is exactly the same no matter which provider you're talking about and those providers plugins are what I talked about ear earlier in this instance I'm actually running registry. example.com I'll come on to that in a moment /t tell me/ proxmox now the reason I'm doing this is because I've had to deploy the release candidate which I showed before on the GitHub that's the one with some of the braking fixes that hasn't been published yet but when that is published I'll simply need to do this I can delete this first part and then it will go out to the web and it will pull down the latest version so as I said this probably won't work in the future but the process is exactly the same you'll need to just change the source and change the versions to match the version that you're using a bit like in doca where you have composed versions or you might have the latest version of a container Etc you pretty much do a similar thing here you will need to update this Over time however to get this working I've pulled down a local copy I will show you how to do that in this video it's pretty straightforward we simply need to clone the repository download and install go and then make it and then we'll copy it over to this location that I'm showing here this is a hard-coded local location but hopefully you won't need to do this in the future the second provider is what it says on the tin it's proxmox so this should give you a clue about what we're going to have to do shortly we need to go into proxmox we need to create an API token and we need to reference it here this is going to give terraform access to our proxmox cluster because it needs to have authenticated API access to generate the virtual machines you'll need to make sure that obviously any firewall rules are taken care of and whichever VM or machine you're running this from has direct access to proxmox so with the provider set up we can hop into test cloud and in the test Cloud it's a bit long because I've created two different types I'll show you what I mean the first one here you can see at the top is cloud it k3s Master now this is because it's using proxmox virtual machine quemu which is what manages virtual machines on proxmox and then the second one here you can basically just give it a name you want you'll see that this one's called master and this one down here is called worker now all of these are taken from the API available from proxmox I'll show you that now the issue with that is this provider doesn't necessarily support all all of those API endpoints and like I said before those are going to change over time that can break providers and again you'll need to make sure that you stay up toate with the most relevant provider and you may even need to change if they become deprecated I'll show you how to do the update to that in a moment the cool thing about this is we can specify a Target node so in my instance I'm using my proxmox Dell I've got two remember so just change this to the name of your node even if you've got one you're going to have to specify it I've then given it a description so I've just called This Cloud in it auntu because that's what I'm going to use doesn't really matter what you do but just make sure it's relevant to what you're doing for your own benefit I've then given it count three h so what that does is it means it's going to create three of these now that makes sense because we're doing a ha Master node setup so we need three so it will run three versions of this Cloud init and create three virtual machines now it's going to clone and existing Cloud init template and I've created this before in a previous video so go ahead make sure that you've got a cloud init already created on proxmox now it starts to get a bit funny and this is where things might change in the future that's because I'm using a release candidate and it hasn't been stable unless I use exactly some of these variables so the operating system is of type cloud in it that should be pretty self-explain and then we get into the configuration of the virtual machine itself so this is a little bit like when you create the actual cloud in it itself you need to specify all of these parameters we're doing exactly the same thing here so that technically you could create a cloud init machine from a template but you could then give it more Calles than it was originally given so in my instance mine were given two and two sockets so four usable cores but if I wanted to here I could have two sockets and four cores and get eight cores it's really up to you in my instance I've set Numa to true because I've got a dual processor system I've set the CPU to be host so it gets all the functionality of the host CPU memory set to 4 gigs and then interestingly we have this here now this is just because effectively I'm running this in a loop because I've got a count of three and so what that will do is everyone will be named in sequence so the first one will be 01 the next one will be 02 03 you can tweet that to whatever you want moving down we've got the cloud init so remember the cloud init is mounted as an IDE CD drive and that's basically what we're doing here the actual data for that is saved on my nvme so this is a storage pool on your proxmox host typically the default will be local-lvm but change this to anything that you might have chosen next the scoy Hardware so I've changed this to ver iio scoy single and then the boot dis I've set to be scy Z so basically in the block of diss below this bit here I'm saying create my drive give it to scuzzy Z and then set that as the boot drive it's a bit like what you would do in the Bios to choose your boot device exactly the same principle here but just for a VM now after that we've got some of the cloud init variables so here I've got my IP address and again we can take advantage of it being sequential so each time this runs it's going to bump the IP address up by one it's going to use the user of auntu and here's my SSH test key that I've created in proxmox so that when this created I can then connect to it automatically through SSH using my terminal and the keys that I've already got set up next we're going to repeat pretty much the entire process however all I've done here is I've tweaked the name and I've changed the number to Two so we want three Masters and we'll end up with two workers and I've changed the name here to just be worker k3s worker now you can tweak any of these so make them larger have more storage space Etc and I really do recommend you go and check out the API so you understand all of the variables you've got at your disposal this doesn't just have to do VMS it can do all the other things that are available through the API provided that the provider supports it so this provider because it's some of semi- deprecated a few weeks ago it's now being given the kiss of life but things like the bgp proxmox that seems to have more of the API available to it and it seems to be better supported at the moment so now let's jump into how we're going to use this so the first thing we're going to need to do is to hop into proxmox we're going to have to get our API key which I showed a moment ago we're then going to need to install ter form itself onto the virtual machine and I'll then show you how to pull down and clone this test copy in case you ever need that hopefully in the future like I've said you won't need that and then we're going to get into the terraform API have a bit of a play and then we're actually going to deploy this and see what comes out the back so over in proxmox and you can see here I've got selected this is just a VM I've spawn up for this demo but to get this working we need to head over to the data center and we need to click on permissions and then API tokens now I've created here this new one token name terraform and to do that I clicked add then I've chosen the user as root I've unticked privilege separation and then I've given it a name in this case I gave it terraform and when that completes it gives you this here so for example if I create terraform 2 untick that we should get a code that only shows Once so you need to to take a reference of this code keep it safe and you need to add it to your script where I showed you earlier that would be over in the providers and it would be this value here now that we've got that set up there's nothing else that we need to do in proxmox I would recommend you check just to make sure that you've got a cloud init template in this case I'm using this one here 2304 non- KVM and if we go back you can see that I've referenced that in this test cloud in it here and it's called this one here so that's where that value came from change that to whatever you've got so now we've set that up we're pretty much ready to go and terraform is a DOT to install so if we head over to the official website there's a handy command that we can grab and paste into our terminal and so over on their website we can see that we've got different installation instructions for different operating systems the one we want is here Linux auntu Debian or whatever you're using I'm simply going to copy this I'm going to head back now into my VSS code and I'm going to paste that command into the terminal hit return that's going to go away and install now it shouldn't take too long it's quite a minimalist install and once that's completed we should have the terraform installed with the terraform CLI available it might require us to update some libraries so I'm just going to hit okay you might want to reboot just to make sure I'm not going to bother in this video but now we should be able to do a terraform Das version and we should see that we've got that now available so we can see that I'm running 1.7.1 on Linux amd64 great now in an Ideal World I should just be able to run this script and it's going to go and pull down this provider now what we typically would need to do is simply a terraform init that is initialize and what that will do is check these providers and pull down that provider so it can't do that because I've specified this one here registry. example.com and that doesn't exist if I change this to the official one which again isn't working at the moment so if I save that and I rerun that command it should go away now and pull it so there you go you see it's pulled down 2.94 which on their GitHub is the latest version however this script here won't work because this is using the format for the release candidate so how do we fix that this might be useful for later on well we're going to have to hop back into the command line and we're going to have to do some installation and makes so the first thing I'm going to do to make this is to pull down and clone the GitHub repository and I'm also going to create just a test folder this terraform provider proxmox so when I've cloned that down all of the files necessary to build should be there now unfortunately I can't click make because I need to install go because this is written in go and so to do that it's a pseudo app update and then an install goang so after this is done we should be in a position to be able to make and so now with that completed and we're in the right folder we need to install make so pseudo app install make once that's done we should be able to run the command make and we should be able to build this release candidate of the provider so if we run make that's now going to go away and build the provider and I'll see you on the other side and so now that's completed we need to create a directory that it expects plugins I.E local ones to be in and so that is terraform DOD in your home directory SL plugins SL and we're calling this one just registry example.com slate proxmox version one and then we need to put at the end Linux amd64 now that I've created this and I'll drop a link in the video description for where to find these instructions we're ready to copy over the binary we created in the previous command and then we should be good to go so now if we look in the folder we were in and then we go into the binary the bin you can see that we've got this file here this is the release candidate so we need to get this folder here we can copy it and then we need to go into the folder we just created here so that was terraform D plugins registry tmate proxmox and it's this folder here so hopefully now if we paste it into this folder here you can see that it's there and now when we run the terraform in nit again we want to go back to here I'm going to undo on this command so it points to registry. example.com I'm going to save that and then back down in here if I do a terraform init it should now go and find that it would be handy if I was in the right directory so I'm going to go back and now if I if I do a terraform in nit it should go away and find it so here you can see it's finding registry versions matching one and it's installing it here so now it's installed the one we just compiled using the latest version that release candidate one and it's not using the one that's available that 2.9.1 4 excellent well what do we do now well we're pretty much good to go and I know that because I've run this earlier but there's a few things you might want to do first so one thing that's really important is something called terraform plan so this is basically a dummy run so let's have a look at that so I've run that and it's going to say now what it's going to do when I actually run this so this should be pretty straightforward and it's quite long that's because it's creating five virtual machines but here you can see is creating the first virtual machine the second the third the fourth the fifth Fifth and at the bottom it should say it plans to add five so there's zero changes and zero to destroy that's because it's queried what's on proxmox already and there's nothing there that needs to change or be destroyed that's because we've not created it so what do we do now well now we're ready to do terraform apply so now we are applying this config that terraform file to my proper MOX so I'm going to hit return here and when you do that you're going to be asked to confirm this there is an option to do this silently and I'm going to hit yes now I'm going to quickly say yes and then alt tab into proxmox you'll see here that it's creating and then if I go over to my proxmox you can see here that it's creating those machines so now we've got Master 03 worker 01 Master O2 Master 01 and if I go go back to vs code you can see here that it's being created on the left it's still creating them and if we actually look at one of these machines so let's take the master for example if I go to the hardware you can see that that all looks right with the credentials and the specification that I put within the terraform script and it's also using this cloud in it here now there's a few little idiosyncrasies with this I think just because they're still doing the ething on the release candidate one for example the cloud init is in ID3 and when I've run this I've also got this spare IDE drive I'm not sure what that is and I've also lost the ability to have SSD equals true on this and also the discard option um I'm hopeful that will be fixed but if I actually hop into this into the console so you'll see that they are automatically deployed and started running hopefully in a moment this will pick up what's actually being put out and we'll see this booting up as normal hopefully I can then log in and I'll have access to it as though it was created simply by right clicking here and clicking clone now if I go back into vs code you'll see here that it said apply is completed resources five were added and that's correct you can see that over here I've now got my entire k3s cluster up and running well at least the infrastructure is up and running and that's right you can see over here I've got the five virtual machines running k3s isn't installed yet but as we know this is the precursor and if you were to now combine and run the script you get the point you've automated your virtual machine installation and then you can automate your k3s deployment on your automated infrastructure how powerful is that eventually you can probably see where I'm headed I want to have a script that I can run that will run terraform it will deploy the infrastructure and then there's anable and it would go and deploy the cluster that means we would have a single click solution for a Bare Bones installation of kubernetes across High availability so hopefully that video was useful in taking the sting out of learning terraform it's a really powerful tool that you can use to automate the provisioning of new infrastructure within your home lab and it's a valuable tool to understand for the Enterprise now as I've said and I'll stress it again this probably won't work longterm in the future the apis will change the providers will change and you will need to stay on top of the configuration files that's something you'll need to manage in own time I will try to keep the GitHub up to date and I do welcome some PRS to help me on that and do not that there is a lot of work going on at the moment to try and get this plug-in back alive and out of that release candidate stage anyway thank you very much for watching I'm going to be using this in future videos to hopefully one day bring you the single oneclick installation of of kubernetes that'll be creating your infrastructure those bare VMS to installing k3s or rk2 through anable anyway if you've liked this video give it a subscribe hit that like button let me know in the comments below if this is something that you're going to use and I'll catch you on the next one take care [Music] everybody

Channel: Jim's Garage

Views: 46,149

Keywords: terraform, terraform tutorial, terraform tutorial for beginners, what is terraform, terraform modules, infrastructure as code with terraform, terraform fundamentals, infrastructure as code, proxmox, proxmox create VM, proxmox tutorial, proxmox setup, kubernetes, docker, docker swarm, linux, vs code

Id: ZGWn6xREdDE

Length: 26min 22sec (1582 seconds)

Published: Wed Jan 31 2024