Before I do anything on Proxmox, I do this first...

as you know my current choice for a hypervisor is proxmox i must have installed proxmox at least 20 times over the last year and every time i go through the install i forget a couple of things so i started creating a list of the things i do every time i install a new proxmox server and i figured i'd share it with you [Music] hey welcome back so i'm techno tim and today we're going to talk about the first things i do after installing proxmox and if you end up having questions about anything in this video or any of my other videos check out my live stream i stream on twitch every tuesday thursday and saturday and i spend a lot of time answering a lot of your questions so you're welcome to join if you run into troubles and thank you ahead of time for the likes and the comments it helps me understand if i'm on the right track so let's get into it so like i mentioned i've installed proxmox many times and each time i reinstall for a different reason either because i learned something new or i worked my whole entire install now this is the list that i've had that i follow every time i set up a new server and it helps me remember to do things consistently and while i've kept this list tucked away for my own use i figured i'd share it with all of you so i put together a list of items of all the things that i do every time i install a new proxmox server some of them are pretty easy and obvious while others are a little more advanced and so let's dive into my list the first up is probably the most obvious one but i won't count it it's installing the latest version of proxmox now you can't really apply any of these if you don't install proxmox but i typically grab the latest install of proxmox after i grab the installation media i boot up i install it i fill out some necessary information and then i boot it up for the first time so once we're at this point we'll jump into my first one first i enable updates now you have two ways of getting updates you can get a subscription to proxmox and get the most stable ones or you can turn on the unstable ones i found that installing the unstable updates ever since proxmod 6 has actually been surprisingly stable now it's up to you whether or not you want to do this but i've had pretty good luck with them so the first thing you want to do is ssh into your proxmox server next we're going to edit our sources.list for aptitude in here we're going to add a line for the non-production updates for proxmox once we add that we'll save it next we'll want to edit the enterprise list and here we'll comment this line out now you only want to comment this line out if you don't have a subscription and the reason why we're commenting this out is so that we don't get errors when we try to update so let's save this next we'll want to update our list with app dash get update then we'll want to do a distribution upgrade with app disk dash upgrade and as you can see i'm up today if you get prompted here to update them just say yes and after this it's probably a good idea to reboot and if you don't want to do this in the terminal we can also do this in the ui if we go into our proxmox server then we click updates and we click refresh here you can query for additional updates and if you have some here you would just click upgrade on the screen right here but as you can see my system is up to date next up is configuring storage for this machine i'll go into my server name and then go into disks and as you can see here i have five disks now this disk right here dev sde is my proxmox installation but these other four are going to be my vm storage so if we want to create a new zfs pool here's how we do it so first for that server you'll want to go into disks then we'll want to choose zfs here we'll create zfs and after clicking that we can see we have no unused disks now this is common if your disks already contain partition information or they've been initialized for many reasons so before we do that let's clear out all the information on those disks so let's close out of here and we'll go back into disks and we'll take note of the device names so i have four devices i want to clear out that's sda sdb sdc and sdd so let's ssh back into our server and i'll run fdisk dev sda once we're in here i'll hit p to see partition information and we can see i have a partition then i'll hit d to delete partition information here i'll say 9 and let's check p again for partition and i'll hit d again and i'll hit p again and now we have an empty disk so we'll want to save that with w and we'll want to do this for all four disks so let's do this three more times and after we've done all that we can go back to the proxmox ui and if we reload our disks page we see that we now have the option to initialize these disks but we don't want to do that so let's go back into zfs let's create zfs and now you can see i have all four disks available so let's name our pool i'm going to name mine fast10 so the reason i'm naming this 10 is fast because it's ssd storage and 10 for the raid level so i'm choosing a striped mirror or a traditional raid 10 or raid 1 plus 0. this gives me a combination of performance and redundancy so we'll change the raid level to raid 10 then we'll keep the compression on and the a shift to 12. and these are recommendations from proxmox next we'll select all disks and choose create so now we created our zfs pool called fast10 and if we look at the details we have a striped mirror so this is good now you can change the zfs config to match your needs and the process is almost exactly the same so let's close out of here next is something i just checked to be sure and that's smart monitoring if we go into disks you'll want to make sure that smart monitoring is turned on and this is the default setting but you might have accidentally turned it off and you want to be sure that these are passing obviously and if we go in here you can see the smart values and if you want to do this in the terminal you can check by running smart ctl a slash dev and then your device so mine's sda and you can see that this one's turned on and you want to do the same for the rest b c and d and again this should be turned on by default but it's always good to check to make sure that this is running so we can run smart checks and email the administrator you if something goes wrong the next thing i do is turn on i o mmu or pci passthrough this is so i can pass through devices from the host machine to the guest machine so that they can take advantage of that hardware now there's a couple things to note before you do this one you'll have to have a processor that supports iom mmu next you'll have to have a motherboard that supports iomu and last you'll have to make sure that this is turned on in the bios of your motherboard now you'll need all three things for this to work and also the technology is named different for intel versus amd and a lot of motherboard manufacturers name this option different in the bios so you'll want to check with your motherboard manufacturer to see how to turn this on but once you've done all that we'll ssh into your proxmox server and in here you'll see a line that says grub underscore cmd line underscore linux underscore default equals quiet what i normally do is comment this line out and add another line which adds another flag intel underscore iommu equals on now if you're running an amd processor you want to change this right here to say amd iomu equals on but once you've got that set you want to save it then you want to run update grub then we'll want to edit our modules then you'll want to add a few lines here so you'll want to make sure you have these four lines and they're all for vfio once you have that we'll save it and then we'll close out and then you'll want to reboot your server now there are also some advanced things you can do in there like interrupt mapping where you can divide up individual pieces of hardware and assign those to virtual machines but i don't have a need for that if you do there's a good proxmox wiki on how to do that now the next thing i normally do is make my proxmox server vlan aware now this is a pretty easy setting but it's also something i do every time i set up a proxmox server so in order to do this we'll go into network we'll select our bridge then we'll take this check box that says vlan aware and then we'll hit ok and just like that our nic is vlan aware and you can restrict this nick to different vlans if you wanted to you would just edit at sea network interfaces you would find this bridge vids and you would change this to the vlan you want it restricted to in this case vlan 10 then you would save it and reboot your server but i'm leaving this open to all vlans and i'll restrict them per virtual machine but the choice is up to you and then you can apply a vlan to the nic on your virtual machine it's as easy as going into your virtual machine going into hardware going into your network device and in here setting the vlan tag and this will be your vlan id it could be 10 could be 20 or whatever your vlan id is but once you have that set now your virtual machine will be communicating on that vlan now the next thing i do is add an nfs share i use this share for backups for isos for a lot of things but it's pretty simple to set up first you'll have to have your nfs share already set up and i used truna so that was pretty simple but to add it is even simpler to do that we'll go at the data center level we'll go into storage and we'll click add nfs here we'll give this an id i'm going to name this backups then we'll choose the server then if your nfs share is set up correctly you should see it here next we'll choose the content type so i'm going to choose disk image iso image vz dump backup file and that's it for me if you want to choose more options you're free to and you can choose your nfs version here but i usually keep it as default and then we should be able to add our nfs share and now we'll see our backups directory and now we have the option to backup and restore our disks and now that we have an nfs share now we have my next one which is scheduling backups so to schedule backups it's pretty easy since we have a share set up already but even if you don't have a share you can do a local backup but to schedule backups we'll go to the data center level then we'll go into backup and we'll click add so this is creating a backup job or a backup schedule so in here i'm going to choose my node then we'll choose our storage and this is the backups here we just created you can choose the day of week you can multi-select here if you want every day then you can choose the start time here i usually choose an off-peak time for me then you can choose whether or not to send an email then you can choose whether or not you want to send an email always or just on failures you can choose your compression level i typically keep this on zstd which is fast and good which is also the default you can choose your mode whether or not you want to suspend stop or take a snapshot then you choose your virtual machines after that we just click create and we have our backup job and the first thing that i do after creating a virtual machine and it's in a good state is create a backup so we just go into backups click backup now choose your storage location choose the mode choose the compression choose whether or not you want to send an email and click backup and that will then back up that device then you can rely on scheduled backups to back up the rest the next thing i do might be pretty obvious but i forget about this every single time until i need it and that's uploading the kvm driver disk for windows it never fails whenever i create a new proxmox server and then create a new windows virtual guest i always forget about the disk so this is something i add to my list that i do right away so that i don't have to worry about it so i grab the windows vert i o drivers i usually download the latest stable then after downloading them i'll upload them to storage and then i'm good to go next i usually upload my windows iso and my linux distribution which is typically ubuntu that way i have these when i need them the next thing i do which is pretty advanced is create a nick team now in order to do this you'll have to have more than one network interface card or nic and the reason i do this is for two reasons first i get some failover that helps out if i accidentally unplug a cable and the rare case that the port dies either on the machine or on the switch and on the other end i get some more bandwidth now this one's kind of tricky to explain but what i found out is that i'm not actually doubling my bandwidth per se but doubling my lanes so essentially with two gigabit nics i end up with two one gigabit pipes and i don't end up with a total of two gigabits per connection i end up with two lanes that have a gigabit each and so this relies on some switch configuration as well as some configuration in proxmox and so i'm going to use the link aggregation protocol or lacp this is part of the ieee spec which is 802.38 now this might be confusing but the best description that i've seen is like driving down the highway so with link aggregation or some people call it lag i don't increase the speed limit on the highway i just add more lanes to the highway and so while the speed limit is still one gigabit i get more lanes to drive one gigabit along with some failover and so for me this opens up some more bandwidth because when i run backups or migrations or restores i can saturate a gigabit link which leaves very little bandwidth for the rest of the traffic and so i end up setting up lacp for this and i'm not a networking expert this is basically a software engineer's perspective of how this should work and so if i'm wrong let me know in the comments section below but like i mentioned there's some stuff you need to do on your switch to make sure that your switch supports it and there's some stuff we need to do on proxmox so i'll really quickly cover the switch because you may not have the same switch that i do but i'm using a unifi switch pro and what i ended up doing was plugging in two network cables to each of the nics on my server once i had those plugged into the server i ran two network cables up to the switch now at least in the unifi pro it's very important that these ports are physically next to each other something i didn't expect and i don't understand why but in unified pro they have to be physically next to each other so if you see here my port 18 and 19 are next to each other and these are two nicks that are coming from my proxmox server so in here you'll want to click edit on one of the switch ports remember one of the two that we want to team we'll click profile overrides then we'll scroll down and we'll change from switching to aggregate and here's why i said that they need to be physically next to each other because we can only specify a port range so i had to do a whole bunch of moving around last night but in here i want the port range to be from 21 to 22 and then it's as simple as clicking apply and this is going to differ depending on your switch but the idea is still the same now i showed you how to do that on the switch before showing you how to do it on proxmox which can be a problem because you might have just taken your proxmox server offline so you might want to temporarily undo that until we get proxmox configured or if you have physical access or ipmi you can remote in that way once you're in your server you'll want to edit your network interfaces and in here you'll see the vlan settings that we set up now the reason why i set up vlans ahead of time is because it makes it a little bit easier to copy and paste that config to your new bond after we created in proxmox so a couple things to check in here first you'll want to make sure that you see both of your interfaces now miner eno1 and eno2 and next you'll see our bridge with a static ip but we need to actually create a linux bond here so i'm going to comment all of this out then i'm going to create a linux bond now we can see this interface that called bond zero inet is set to manual then we're going to bond this to two devices you know o1 and eno2 next we're going to set the bond memon to 100 now this is bonding monitoring and i'll be completely honest i don't fully understand it but this monitors the bond i guess but this is the setting that proxmox says to use next is our bond mode and this is what i was talking about earlier we set it to 802.3a and this is for lacp and next is bond dash xmid or transmit and then our hash policy and so our hash policy is two plus three and admittedly i don't fully understand this it seems to be a hashing policy that the bond uses to determine where to send the packets if you have a better explanation than that let me know in the comments below so after that set now we'll just create a bridge so we'll set auto vmbr0 for our bridge and if you'll look at the section i commented out it looks almost exactly the same so if we look we have a static ip and here's our address here's our gateway and then our bridge ports are now actually the bond instead of the nick because before we had a bridge port of eno1 but this time we're actually binding it to that bond we just created before this and so this is all going to look the same then after this and these are the settings for our vlan awareness and that's why we set up a vlan before so ultimately what this means now is that we have a nick team setup that team has some failover it also has some increased bandwidth or pipes and then it's also vlan aware so after saving that you can either bring your network interfaces up and down or just reboot your server now the next thing that i do is actually create a template i typically spin up a few ubuntu servers in my environment now after i've set up one and followed all the steps that i usually follow when setting up a linux machine i then convert it to a template this allows me to clone this machine very easily and it's very easy to do basically once you have your machine shut down you want to right click it and convert to template you want to say yes here and now it will convert it to a template so worth mentioning after you convert this to a template you can't use this virtual machine anymore but you can spin up new virtual machines really fast and you can do that by right-clicking and clone now you have some options here you can change the vm id you can change the name you can set the resource pool but then you have a mode that you set it's either linked clone or full clone now link clone is going to be a link back to the original template it's going to use a lot less disk space because it's based off the original clone and if you do a full clone it's a standalone clone and so the differences are very nuanced but when i do this i usually choose a full clone that's because i don't want to have to worry about this template that's out there that i'll have to keep forever and so after creating a full clone and cloning it it will create a clone of that virtual machine now i've noticed on ubuntu there's a few things you need to do afterwards now these things i think should work in proxmox but they don't and so those things that you might need to change afterwards are things like your host name your host file your machine id which is your unique id for the machine and your machine id which is your unique id for your machine this one was a tough one to track down because if your machine id doesn't change you'll end up having the same nic on that virtual machine which will end up getting the same dhcp address if you have a reservation and so i'll have all of those commands that you need to run in the documentation below and if you're going to clone a windows machine you might want to run a sysprep utility ahead of time now i'm not going to go into detail about running sysprep and shutting down the machine but that basically prepares it for an image or a clone and if you run into anything else you need to change along the way let me know in the comments section below because these things are really hard to track down and this is something where maybe installing these operating systems unattended might be a little safer but maybe in a future tutorial now the last thing i do is kind of a bonus because not everyone may need to do this but after creating a new proxmox server you might need to add this to a cluster and this is pretty simple to do but there's some things you need to know first first you want to make sure that your hostname is set these can't be changed later on next you'll want to make sure all of your networking is in place again this can't be changed later on the next thing you'll need to make sure is that you don't have any virtual machines when you add a proxmox server to a cluster it can't contain any virtual machines so let's delete this virtual machine here then once you've done that you'll want to go to your first proxmox server or the one that already has a cluster created then you'll want to go into the data center section you want to go into cluster and you'll want to click create cluster then we'll name this cluster then you'll want to name this cluster then you'll want to choose the nick that this cluster will communicate on then we'll click create once it's created we'll close out then we'll click join information and we'll want to copy this information to the clipboard then it's as simple as going back into your new proxmox server going into cluster joining the cluster and pasting our cluster information then click join then after joining the cluster you can now see both of your proxmox server within the data center view and you can view this from any server within your cluster now worth mentioning i'm not running this in ha to run this in ha you need an odd number of servers that's because you need to establish quorum and quorum is established when all nodes vote and if you have an even number of nodes no one can ever win so that's why it's important to have an odd number of nodes for quorum and for ha now i've seen some interesting projects where you can run coral sync on say a raspberry pi to establish quorum but for me personally i don't need aha i just need an easy way to migrate virtual machines back and forth and then i'll end up building high availability into my kubernetes services and cluster there which is a good segue probably into a later video and so that's everything i do when i spin up a new proxmox server this is a list that i've added to over the last year over many proxmox installations and instead of keeping it to myself i decided to share with all of you so is there anything i missed anything that you do when you spin up a new proxmox server is there anything you would have done differently or not at all if so let me know in the comments section below and while you're down there don't forget to give this video a like and consider subscribing if you haven't already and if you have more questions you can always join my twitch stream i stream every tuesday thursday and saturday so if you have a question about this video or really any of my videos hop in my stream and let's figure it out and again thank you so much for watching and until next time stream on my friends i can't thank everyone enough for all the help i get in the discord server it's it's fantastic so thank you all so much for for helping in any way even if it's just pointing someone to a channel giving someone help i mean i've seen people fire up whole entire virtual machines and spin up a whole entire environment just to see if they can get the same problem and you won't find that kind of help anywhere anywhere and somehow i was lucky enough to get those people on my discord server so i really appreciate it heartfelt thank you because i i see it every day and i it's it's heartwarming

Channel: Techno Tim

Views: 247,683

Keywords: proxmox getting started, techno tim, homelab, home lab, proxmox updates, proxmox tutorial, proxmox striped mirror, zfs, raid10, VLAN, VLAN aware, LAG, nic teaming, upload ISO, aggregate, LACP, 802.3ad, NIC bond, clone vm, proxmox clone vm, proxmox cluster, install proxmox, how to set up a share in proxmox, proxmox nfs, proxmox backups, how to backup proxmox, how to backup proxmox vm, proxmox initial backup, proxmox vm create clone, how to fix clone proxmox, vm, virtualization, raidz

Id: GoZaMgEgrHw

Length: 23min 4sec (1384 seconds)

Published: Sat Nov 28 2020

Reddit Comments

[u/airpuffin]

I recently discovered your YouTube channel after getting into homelab-ing this past Christmas. I love your content and your videos, especially this one which has helped me get off the ground with the “what do I do now” of my homelab. Thanks for taking the time to show off the cool things you do!