All about Proxmox Boot drives: Capacity, Endurance and Performance

today i'm going to be taking a look at proxmox boot drives what type of drives work best how much endurance is needed how large the drive should be how you should format the drives and other little things about what makes a drive work well for proxmox boot drive and what type of issues you can run into if it isn't set up correctly the first question to take a look at is do you want to have a separate boot drive in proxmox or do you want to have it has the same as your data drive proxmox can be set up both ways where all the data is stored on the boot drive so all the vm images iso drives templates and the boot data stored on a single disk or multiple disks in a raid configuration or you can have it so the boot drives only store boot data or like boot data and iso images and the data drives only store the data that the vms need or other types of files on the system you don't even also need a local data drive you can also store data on network storage i typically like to have servers where my data drives are separate from my boot drives i think it makes it easier to manage as those separate devices and if something fails i can have all my data drives fail and i can still have my proxmox system boot and i can manage it and try to fix my data drives some systems or setups may require you to have a separate boot drive so for example some of my servers have the data drives on an hba that isn't bootable in that specific system so in that case i can't have those drives as my boot drives even if i wanted to and if you wanted to use network storage over a san you often can't have that as the bootable drive one other thing to note is if you are having a vm storage and your boot drive on the same disk you need to add the requirements of both so typically the boot drive doesn't need very much but just make sure that you have enough storage and iops and all of this for your data drives and your boot partition most of what i'm going to be talking about in the rest of this video is going to apply to just the boot drive so take your needs of your data drives and add that to what i'm talking about here another thing is i'd say the boot drive is probably the most important in a proxmox system because if it goes down essentially the rest of the system is dead but having a shared boot and data drive can make a lot of sense in some scenarios where you're using like a smaller pc where you may only have one or two drive bays that you want to get the most out of even with something like this one use super micro server that only has four drive base if you're trying to get the most storage you can that extra drive bay being used for data can make a large difference having a shared boot and data drive also saves the power and the cost of an additional data drive which can help save costs especially on a lower cost system the exact way that you're using your system kind of determines if a separate boot drive would make more or less sense typically my thought is the larger the system and the more drives and data it's going through the more it makes sense to have a separate boot drive if it's a small low-cost system combining the boot drive and the data drive seems to be it makes a lot more sense in that case now let's talk about file systems for the boot drive the boot drive really isn't that intensive in terms of features or i o or maximum storage capacity so any of the file systems that proxmox provide will work fine traditionally i prefer zfs because it gives me the option for a mirrored boot volume and none of the other file systems on proxmox let you do that this lets you have a raid1 boot volume even on systems or controllers that don't support it with hardware raid and i find it generally works quite well proxmox has a nice guide to restore a failed disk if that happens to you and i haven't had any issues with it i'd stay away from dtrfs because it's still in beta and i just want my boot drive to be stable ext4 and xfs also work quite well you just can't do raid which i like to do now let's talk about the amount of capacity needed for a proxmox boot drive so proxmox installer requires at least eight gibby bytes of storage to even let you install it on a drive and it really doesn't need much storage space compared to a lot of other os's like windows a base installer proxmox on the slash or root volume only uses about 1.7 gigabytes once fully updated you need up also another about half a gigabyte for boot volumes and kind of the boot sector and partitions and then beyond that you want to have a good amount extra space for other things that come up one example is i have kind of two main proxmox servers that have been running for roughly about two years since their installation now and been running basically 24 7 with over 20 vms on both of them one of my servers i kind of play around with a bit less and has used about 2.5 gigabytes of space on its boot volume and the other one i have done a little bit more tweaking has more logs running and it's using about 7.5 gigs of space on its boot volume i have a lot of logs going into my main system logs and i also have atop logging system performance which uses a good amount of storage after quite a bit of time i also on that system have some older versions as it likes to keep older kernel versions and kernel modules by default in the system so those will slowly build up over time but even then five gigabytes after about two years isn't very much space so if you get a larger drive like i'd probably suggest getting a 64 gigabyte drive your system will have to be probably at least 10 years old before that drive fills up even kind of in my worst case scenario and while the boot drive is traditionally only used for boot files some things like when you upload an iso file in the web browser it copies it to slash virus temp first and then copies it to the actual data store so you need to have enough space for things like your iso image so trying to have maybe 10 gigs extra space is always nice to have on that system the other advantage of larger drives like a 64 gigabyte drive instead of a 16 gigabyte drive is normally larger drives a bit faster and they can handle more endurance and a more reliable so it's kind of a might as well and the extra additional cost for most drives between 16 and 64 gigabytes is very minimal now let's talk about the amount of endurance needed or how much rights are being done by proxmox onto the boot drive so on my systems that have a separate boot drive i see roughly 30 gigabytes a day of data being written to the boot drive and this seemed a bit high to me so i actually wrote a little script to look at average writes per day and i'm going to try to link that below in a github just to show if you'd like to play around with it and see how much data your drives are writing every day on average i've also googled on the forums and the 30 gigabytes per day number seems to be about correct from what other people have said so this seems to be in my ballpark in my example my kind of lesser use system has about 28 gigabytes per day and my more heavily used system is about 42 gigabytes per day the 30 gigabytes of data and writes equates to a little bit more than 10 terabytes per year of writes on your ssd one thing to worry about is things like usb flash drives these can often have significantly less endurance than our standard ssd so i'd probably want to stay away from it this is doing quite a bit of logging and writing on the boot drive compared to other os's which essentially just read from it and stored in ram so you probably don't want to use a flash stick if you don't have to for the boot drive mechanical hard drives can also handle this amount of rights just fine without any issues taking a look at what's doing the rights from what i can tell it's probably the coral sync process which is what's syncing the data that proxmox uses to keep all the nodes in the cluster in sync and it seems to be essentially writing this status all the time to a little file now let's talk up a little bit about performance needed for a proxmox boot drive i did a little bit of testing between a traditional desktop mechanical hard drive and a sata 3 ssd in this case an 850 evo and realistically the performance differences weren't that big i ran three different tests a proxmox install and update to all the latest packages and then i also did a reboot test for my usage the additional i o performance really isn't needed for proxmox boot drive and during normal day-to-day use like creating vms updating vms viewing performance it's essentially the same though extra performance is always nice to have and often with the boot drive options you have which i'll look at in a little bit you aren't paying much more for a lot of extra performance now that i've gone over all the different attributes that a boot drive and proxmox has and how they should be set up let's kind of go over some classes and devices and how well i'd recommend or think they'd work for proxmox boot drive so the first thing you're going to look at is like an ssd so with a lot of modern servers you actually have m.2 slots or extra two and a half inch slots so it can be quite easy to put an extra ssd in your server and ssds are going to be great for boot drives in general they're very reliable they're pretty low power low capacity drives are quite cheap and you're not really giving up very many spaces to put them in so you can put in like two boot drives in your system and still have plenty of data storage drives in the front for putting vm and storage on a ssd like either a sata nvme ssd seems to make the most amount of sense for me i'd probably go 128 gigs here as you don't need the extra space if you aren't storing anything other than boot drives on one thing i would recommend doing is trying to get your ssd from like a large known brand so like something like a samsung out there or another large one instead of a smaller brand you spend a little bit more for the large brands but for most of the reports and data i've seen those larger brands are going to be more reliable which is the most important part of a boot drive another way you can possibly store boot data on a system is via satadom or a similar kind of dedicated boot data in your server i just make sure that these systems have enough right endurance for an actual drive being used for example super micro labels theirs as one drive write per day which is more than plenty for a proxmox boot drive unlike the 64 gigabyte model but you probably don't want something that's based off of like sd cards as these often built with products like esxi which are almost running the drives as read once during boot up and never touch it again and store all the log data on additional drive these can be a great solution for boot drives as they take very little space in the server very little power and you basically aren't giving up any data drives so if i put a little sata dom in my server here i still have all the front data drives to put ssds or hard drives for my data whereas if i was using the front bays then i might be giving up two front bays for a mirrored boot drive and then only have two more bays in the system to store my data drives one other option for a proxmox boot drive is a mechanical hard drive these will work fine as a boot drive but i wouldn't recommend buying one in a new system as an ssd of 128 gigs is normally going to be cheaper lower power and more reliable but if you have the hard drives laying around they will work perfectly fine and i believe a mirror of mechanical hard drives is going to be plenty in terms of reliability for most uses of a proxmox server one other class of device that you could use as a boot drive is things like a usb flash drive or other low-end flash devices such as like a low-end sd card these can seem very appealing because they definitely won't take a data bay most servers have multiple usb ports so you can run them in mirrored configurations if you want to and they're very cheap but also coming with that cheapness they're not very reliable typically and since you don't want your boot drives to go down because it takes down the rest of your system i'd say paying a little bit more for a real ssd to get a bit more reliability especially when you're also already spending quite a bit of money on the rest of the server makes a lot of sense but i do have one of my personal proxmox servers running on a raid one of two 16 gigabyte flash drives without any issues thanks for watching my video going over different boot drive options for proxmox server let me know what type of configuration you're using for your proxmox boot drives below and how it's working for you

Channel: ElectronicsWizardry

Views: 28,100

Keywords:

Id: lhRPhBDysVI

Length: 11min 34sec (694 seconds)

Published: Fri May 27 2022