Proxmox Automation with Proxmox Helper Scripts!

Video Statistics and Information

Video
Captions Word Cloud
Reddit Comments
Captions
Today we're going to take a look at this wonderful little open source project called Proxmox Helper Scripts. Proxmox Helper Scripts helps you automate many of the common tasks you do with Proxmox. For instance, setting up the test repositories, removing some of the nag screens, cleaning up old kernels, and lots of things that are focused on LXC containers. If you're not familiar with LXC containers, think of LXC containers like lightweight virtual machines. They provide OS level virtualization, but they share the kernel with the host. Whereas with virtual machines, they're completely isolated from the host, so the kernel is actually part of the operating system within the virtual machine. There's no sharing. And this also differs from Docker a little bit, although we're not going to talk about Docker today. But Docker does an application level virtualization that runs on top of the Docker engine. But we're not going to talk about Docker today, and we're not going to talk about virtual machines today. We're going to talk a lot about LXC containers because the Proxmox Helper Scripts has a lot of scripts to help you spin up applications on your Proxmox host, which is pretty awesome. So that's what we'll cover today. We're going to talk about Proxmox Helper Scripts. We're going to run some of those scripts to help us automate some of the things on our Proxmox server. And then we're going to run lots of scripts to spin up applications in LXC containers. And then we'll use some scripts to help us keep those LXC containers up to date, and even a script to help us back up a Proxmox configuration. Sound good? Let's give it a shot. Worth noting, I'll be running all these scripts on a clean Proxmox server that I just built, running 8.2.2. And the only thing that I've done is install Proxmox and configure some storage. So let's take a look at some of the scripts that we can run. So if we go out to helper-scripts.com, you'll see this Proxmox VE Helper Scripts web page. If we browse scripts, we can see a lot of these scripts are broken down by category. First, we'll see scripts that were newly added to this repo. So we can see some VE tools. These are ones specifically for the Proxmox host. We'll see a few different home assistant installations. We'll see some things grouped into automation, MQTT, into databases, into ZigBee, so on and so forth. And as you can see, there are many more applications listed under all of these categories. Now we'll be diving into some of these here in a few, but I wanted to focus on the Proxmox VE tools ones first. Now, like I said, I just spun up a brand new Proxmox virtual server. And one of the first things I'll want to do is this VE post install. So what does this script do? Well, it's all the things I normally do after I install Proxmox. And if we go into the script, we can see this script provides options for managing Proxmox VE repositories, including disabling the enterprise repo, which I do, adding the correct PVE sources, enabling the no subscription repo, which I always do, adding a test repo, which I don't normally do. I might disable that. And then disabling the subscription NAG, I kind of do that. And then it will update Proxmox VE, and then it will reboot the system. And like I said, I didn't do any of this yet. If we look at my Proxmox server and we go under the updates and the repositories, you can see I still have the enterprise enabled. I also have the NAG screen if I were to log out and then log back in, you're going to see that NAG screen right here, the no subscription NAG. So how would we run this? We would copy and paste this script right here, but this is probably a good time to remind you that running scripts from the internet is not a good idea in general. You have to trust the script, and I would highly recommend looking at the script and understanding what it does. And this is no exception. While a lot of people trust this website and trust these scripts, we've been running them for a long time, it's still a good idea to make sure you understand what this script does. Now you can see it's just going to run a bash wget. So it's going to get this from GitHub from raw/main/misk, and then it's going to get this postpveinstall.sh. And that's here on the GitHub repo. So if you wanted to look at it, we would go into Misk and we would look for this postinstall pve, it's right here. And this is the script it's going to run. Now the way that these are set up, it's always going to run latest. There are ways to actually pin this to a commit tag and make sure that you're running the script out of one of these commit tags. Not going to cover that here. Maybe I'll put it in the documentation, but for the most part you're always going to run latest. So I would recommend just taking a peek at the script before you run it just to make sure that, hey, no one's made an issue to say, hey, it's broken, or someone says there's a security issue with this. I would just recommend taking a peek at it really quick before you run it. So pretend we took a peek at it. I'm going to copy this to my clipboard. And then what I'm going to do is go into my Proxmox VE host, and I'm not going to SSH into it or anything like that. What we want to do is open a shell prompt from Proxmox. What I'm going to do is post this in here. And again, this is going to get this postpveinstall shell script. We're going to run this. Do you want to start this script? Yes, I do. First question, it's going to ask me to correct some of the sources. Yes, I want you to do that. Then it's going to ask me, do I want to disable the Enterprise repository? Yes, I do, because I don't want those errors, and I don't have a subscription. Enable the no subscription repository? Yes. Correct the Ceph package sources? I'll say yes, even though I'm not going to use Ceph. And this is nice. Add the pvetest repository, but disable it. Sure, let's add it disabled because that's what I was going to do anyways. Disable the subscription nag? Yes, please. And this screen is basically saying, hey, if you support Proxmox Work, try to help them and support them financially, maybe with a subscription. Now it's asking me to disable high availability. If you're running a single node, you can do this and free up some resources. If you ever plan to use high availability later, you can re-enable them or you can just say no right here. But on this node, I'm not going to run high availability, so I will say yes. Now it's asking me, do we want to update it? Yes, please. Now it's asking me to reboot the node, which we probably should because there were updates, kernel updates. So it's probably a good idea to do that sooner than later. And now it's rebooting. Okay, so now it's back up. Let's take a look at our servers. Just take a look in the repositories and we can see here, hey, the no subscription repository is enabled like we wanted. We see that the enterprise one is disabled and we can see the test one has been added but is also disabled. So good. We are in a good state. This is exactly what we wanted. Now one of the other things you might want to do as you update Proxmox over time is to do a kernel clean. This is typically, I guess, kind of hard and also a little bit dangerous to do. So if you want to see what kernel you're currently using because you don't want to delete that, you would do a uname -r and you could see the kernel that I'm running is 684-3pve. So if I were to run this script, let's copy it, go into the shell and paste it into here. It's going to let me know that hey, this is used to clean up old kernels, use at your own risk. And here's the only kernel that is old. This is actually the one right before we updated and that's actually nice that it doesn't show the one that's currently in use. Now if you wanted to, you could delete this kernel and if you've been running Proxmox for a long time, you'll see lots of kernels here. I have older installs that probably have 10 to 15 entries in here. What I would recommend doing is probably not deleting the last one or two and keeping those around so that if something happens, on boot up, you can easily switch back to the older kernel. But I'm showing you where it is and how to do it just in case. I would probably keep one or two, especially since I just updated this kernel, I probably wouldn't delete this kernel. But let's do it just for science, I guess. So I wanted to remove that kernel. Yes, it's removing that old kernel. Then it's updating grub and there we go. Now we don't have any more old kernels there. And if we do a uname -mrs, we can see we only have that one kernel now. The next script that I think is pretty important and pretty nice is actually backing up our Proxmox host configuration. This is a general purpose backup script, but we're going to use it to back up Proxmox. So let's copy this out here and paste it in our shell. And so it's telling us that same thing that, hey, it's just going to back up files and folders in a directory that we choose. Proceed. The first question it's asking us is basically, where do we want to back up these files? Now it doesn't make a ton of sense to back up these files to the same Proxmox host. And so I think what they're suggesting, and I think this is a good idea too, is to mount a directory, maybe a share, NFS share, SMB share, basically mounting a network directory and then backing it up to that mounted share. I'm not going to do that because I'm not going to mount a share, just showing you how this works. But I'm going to back mine up to /root/backups just so we can see this happen. The next question is asking us, hey, which folder or files do we want to back up? Now I've seen that /etc/pve is probably a good folder to back up. I think they're suggesting all of /etc, which is probably better. But I think that most of the files that you need to back up are inside of this pve folder, like your configurations for your virtual machines, your network and things like that. But if you want to back up all of /etc, but I typically do /etc/pve. So let's say okay here. It's asking us here, what folder and files do you want to back up inside of this folder? Well, there's only one folder in there, so /etc/pve, and then it's saying, hey, we're going to create this backup of this folder to this location. Hit enter. Hey, it backed it up. And then we're back to the same question again. So we can exit out of here. So let's go to where we backed that up. If we go into /root, I realize they didn't create a backups directory. So it appended the backups to the name of the file, which is totally fine, not important. But you can see it created a backup archive and it stored it here. Now again, if you had this mounted on a network share, then you would be good to go and you have those important files off your server. Now let's move on from kind of maintenance type scripts that we run on our Proxmox host, and let's look at some of the applications here. Now for instance, if you wanted to install Home Assistant, there are quite a few ways to install Home Assistant. As a VM, you can install the Home Assistant container, just LXC, or you can install the Home Assistant core LXC, Pymox HAOS as a VM, or Podman Home Assistant container LXC. So what I'm going to do is install the Home Assistant core LXC, and this is just the core install of Home Assistant. It's typically what I do. And if you've ever used the Docker version, it's that same version where you don't have access to all of the plugins because those are installed through containers and you install them yourself if you need them, which has never been a problem for me because if I need to run those containers, I just spin up those containers rather than telling Home Assistant to spin them up for me. Long story short, I'm going to go with Home Assistant core and I think you should too. And here we see a couple of notifications. This should be run on Ubuntu 2404 only. It requires PvE 684-3 or newer, or way newer than that. And then it's telling us where our config path is going to be. So when that LXC container spins up, it's going to be inside of root, inside of a hidden folder called Home Assistant. So let's copy this really quick and then let's go back into our shell right here. Let's paste it. And that's a common thing I've seen people get confused on. They would SSH into the server and try to run the script. You don't want to do that. You want to run it from the PvE shell right here. So let's do that. Let's run the script. Now it's confirming that we want to actually install this. Yes, we do. Telling me that my shell is not currently bashed because it's ZSH, I switched it. So let me change my shell really quick. So I'm running OhMyZish. So I'm just going to uninstall it really quick. Going to uninstall it. Yes, removed it. Source our bash. Reopen a new shell. There we go. Now we're in bash. Now let's copy that script again. Paste it in here. Yes, we want to create this container. Now it's asking us if we want to use the default settings. Let's actually go into advanced so we can choose some additional options. So it's telling us pretty much we need to use 2404 here. Then we can choose our distribution. Say okay. We need to use 2404. Say okay. We want to run this unprivileged. If you need to use USB, I think you need to use it privileged. I don't need USB on this installation. So I'm going to say unprivileged. So say okay. Then it's asking us to set a root password for this container. Then it's asking us to set the container ID. The default is fine. Set the host name, home assistant core is fine with me. Then it's asking us to set the disk size for this container. 8 gigs is fine for me. How many cores do we want to give it? 2 is typically fine for containers because there shouldn't be many processes running in here because remember, it shares the kernel with the host. I'm going to give it 4, but 2 should have been fine. How much RAM do we want to give it? Going to give it 1 gig. Which bridge do we want to use? VMBR 0 is fine with me. Then it's asking us if we want to set a static IP or use DHCP. DHCP is fine for me. If we want to use apt-cacher, didn't show this, but you could also install an LXC container that will cache all of your apt packages within that container and then you can point all of your additional LXC containers to that apt-cacher so it'll cache all of those updates locally so you don't have to reach out to the internet for all of them. It's pretty cool, easy to set up. I don't have one, so I'm going to leave it blank. Disable IPv6. I'm not going to use it, but I'm also not going to disable it. MTU, not going to touch it. Search domain, not going to add this. It's totally fine. And then it's asking us to set a DNS server. I'll use the host. I don't need to change anything here. MAC address, going to use the default, some randomly generated one that gets assigned to us. And then VLAN, I don't need one there either. Enable root SSH access. This is kind of up to you. I am going to enable it so I can SSH into it, but you can enable or disable this however you want. Enable verbose. Nope, I don't need verbose. And then, hey, am I ready to create it? Yes, I am. There's a lot of questions. But the rest is automated. Now it's going to ask us which storage pool do we want to install this in. Well, I install all of mine into this FAST-10, which is RAID 10 kind of sort of. Basically mirrored VDEVs. I have six SSDs in there right now and they're mirrored pairs. So I'm going to choose that one. So first it's downloading that template, right? So it's downloading that LXC template, which is that Ubuntu 2404. Now it's creating the container. Now it's starting the container. Now it's setting up the container, updating it. Now it's installing all of the dependencies for Home Assistant. Now it's installing Home Assistant. So while this is going, I don't typically click off the shell. I'll wait till it's done because if you click off the shell and then click back, I think you'll get a new shell. And then you really don't know if this is done. Obviously, there are ways to figure out, but I would recommend just waiting here while this gets installed. And there we go. Now it's installed and it says, "Hey, we can reach Home Assistant Core now at this IP address on this port." And so that's the IP address that this LXC container has. So if we go to this LXC container and we go into summary, I noticed that it doesn't always show us our IP address here. I don't know what's going on. But if you wanted to and you logged into here and you actually logged in just to prove and you did an IPA, you can see that, hey, there's the IP address that we just saw. I feel like it should show here. I think this is a bug that it's not because it should show here. I remember doing that in the past. Anyways, let's check out Home Assistant. So we're going to go to that IP and that port. We go there, hey, here's Home Assistant and it's running. So pretty cool. We have Home Assistant up and running pretty quickly. So this is probably a good place to mention how this differs from Docker. If you think about it, what we just did was spin up an Ubuntu LXC container, right? Now we're sharing our kernel with the host, but then we've manually or automatically installed all of the dependencies that we needed along with Home Assistant. So that means that this lightweight virtual machine, this LXC container has all of those dependencies installed on it, right? And so if you go and run an app update dist upgrade and you update it, you're going to update all of those dependencies and that might be what you want and it might not be what you want. Just think, for instance, if you had a dependency like SQLite for Home Assistant and, you know, Home Assistant was only compatible, say, I'm going to make this up. But, you know, anything lower than SQLite 10 and then you run an app update and app upgrade and you upgraded SQLite to something higher, say like 11. Now you just might have broken Home Assistant. So, I mean, it's the same concept you have in virtual machines. You're just going to have to manage all of your dependencies wisely. Whereas in a Docker container, it ships with all of its dependencies. It's bundled up inside of a container image and you pull down and run that image as is. I'm only calling that out because as you'll see later on, we're going to update all these containers and if they ever break, that's kind of up to you to figure out, rather than in Docker, the dependencies ship with the container. Anyways, we're not using Docker so I'm not going to talk about it, but I want people to know kind of, I guess, where the landmines are. So anyways, we got Home Assistant going. That's pretty awesome. Let's do a couple of other things right now. So let's go back to our scripts. We can install a lot of other things now. I'm not going to do it. But one thing I did want to call out is turnkey. So we can install turnkey appliance-based LXC containers. So turnkey is really cool. It's a collection of LXC containers that have applications already installed in them. And you can think of them like virtual appliances. Basically, think about it like a mini virtual machine. They can pull down and run and everything's preconfigured. So it's really cool. And so typically, you could install these in Proxmox, but the Proxmox helper scripts help us do that a little bit easier. So if we wanted to install a turnkey LXC container, let's go to turnkey right here. Let's copy this command. Let's go into our shell. Let's run this. We want to say, yes, we want to install a container. Now it's asking us, hey, which one do we want to create? So I'm just going to pick any in here. I'll try to pick one that's a little bit smaller because a lot of these are pretty big. I mean, Nextcloud might be kind of small. Let's just do it. Nextcloud, I could install all of them, git-t or gitlab, but I feel like that's going to take a little bit. But let's just choose Nextcloud. I'm going to say OK. Where do we want to store the container? Well, I'm going to do it on my fast array. Once a turnkey LXC container is done, you'll see this right here where it says, hey, our login is root and then some password to it. And then you'll see an IP address appear. First, I was a little bit confused on what this was, but we'll need to go to this IP address and then we'll need to log in with this account root with this password. And what we'll want to do is actually SSH into that machine. If you try to go to that IP address, like I just did, you're going to see this helpful screen that says we need to initialize the system and we can't do it without SSH. So it's actually nice they put that there. We'll need to SSH into here and they actually give us the command. So let's SSH into the server. Yes, I want to accept. Now what's the password? It was that password that it showed us in the shell. Now we SSH in. And so now we need to do a few more things. First, we'll need to set up MySQL password for the account adminer. So let's add a password. Confirm. Now we'll need a password for the Nextcloud admin account. Confirm. Now it's telling me the password I tried to use for that admin account is on a compromised password list because it was kind of sort of password. Yeah, it's a demo account. So let's put a real one. All right. Now the domain. It's fine me doing example.com. Now it's going to ask me stuff about using turnkey hub, which I'm not. So I'm going to skip this. Whether or not I want email notifications. Going to skip this too. Security updates. Hey, do we want to install them? I'm going to say yes. Now it's going to update that LXC container. Now it's asking us if we want to reboot this container now. Yes, we do. Should go really quick because it's an LXC container. And if we go back to here, we can see that this container is now here, turnkey Nextcloud. If we go out to the IP address, we can see we have a self-signed cert. We go out to the IP address, we can see we're at the Nextcloud screen. Now there's some other things we need to do, but you can see that it's actually working. Kind of. It's because we didn't set our domain name properly. Not important. What is important is that we got it installed through a turnkey LXC container. So now that we have a couple of containers running, I installed a few while I was going. I have Home Assistant, Nextcloud, PyHole, Homepage, Cloudflare D. And as you start to build up these containers, you want to make sure that they stay up to date. So they actually have a Proxmox helper script for them. So if we go back into the Proxmox VE tools, what we can do is we can set a cron job to update all of our LXC containers. So if we copy this out here, we go back into our shell and paste it in here, what this is going to do is help us set up a cron schedule to update all of our containers. So you can add or remove. We haven't added one yet, so let's add it. Do we want to proceed? Yes. And done. So it's going to do it every Sunday night, I think at midnight. But if we want to go and we want to remove that schedule, we can remove it that easy. Or if we want to go back and we want to add it and say yes, we can add it back. So now it's going to update all of our LXC containers automatically. So pretty cool. If we wanted to run it now, what we could do is run this LXC updater. Let's run this script. Go back to our shell. We'll run this now. And this is going to update all of our LXC containers, basically the same thing that cron job was going to do, but we'll run it now just to see. And it sees all of our LXC containers. Let's select them all. Obviously, you can deselect ones you didn't want to update. And it's going to go through and update them all. And it did update them really quick. So another thing that I noticed in here is that it has this thing called monitor all. And what this does is monitor each LXC container and make sure it's still alive. And if it's not alive, I think it will reset it. So let's run this script right here. And it's asking, hey, do you want to add this monitor to all? Let's add it. Yes, we want to monitor all. And I think we just added the script right there. So I haven't had any problems with LXC containers getting hung up. And I haven't had that with VMs either. But if you do have that problem or you don't know if you have that problem, just want to make sure that you don't, you can actually install this script to where it will reset those containers if they do get hung. So pretty cool. So if you want to support their work, you can do this on their website by buying them a coffee right here. Or you can also go out to GitHub and then going to this coffee link as well and supporting them there. I think it's a great project. It's an easy way to spin up and test LXC containers. And it gives you a lot of helper scripts that help you manage your Proxmox installation. Well, I hope you enjoy Proxmox helper scripts. I hope you enjoyed this video. I'm Tim. Thanks for watching.
Info
Channel: Techno Tim
Views: 67,908
Rating: undefined out of 5
Keywords: techno tim, technotim, homelab, home lab, proxmox, proxmox scripts, proxmox automation, lxc, lxd, lxc containers, linux containers, docker containers, docker, vm, virtual machine, docker vs lxc, lxc vs virtual machine, lxc vs vm, docker vs lxc vs virtual machines, promox ve, helper-scripts, proxmox helper-scripts, open sources
Id: kcpu4z5eSEU
Channel Id: undefined
Length: 24min 14sec (1454 seconds)
Published: Thu May 30 2024
Related Videos
Note
Please note that this website is currently a work in progress! Lots of interesting data and statistics to come.