Setting Up My First Proxmox High Availability Cluster

hey guys this is my new three node proxmox cluster set up with high availability and in this video we're going to talk about how I got it set up so if these boards look familiar they probably should if you've watched this channel before uh those are zema board 832s that were provided to me by Zima board uh months ago I was going to make this video a long time ago and just never got around to it so uh sorry to the folks at Zima Ford for taking so long to do this but uh thanks for providing the hardware I do appreciate that uh so these are 832s these that means it's got eight gigs of RAM and 32 gigs of storage I do have a one terabyte SSD plugged into each one of them there's some cables in the back where you can kind of see uh that plugged in and then I've got a 64 gig USB drive as the boot but with all of that said let's uh let's jump over and take a look at the process of getting this set up uh so that we can then kind of go into with the next video in this little project might be this video is sponsored by tuxedo computers in The Tuxedo Aura 15 gen 2. the aura 15 Gen 2 comes with an AMD ryzen 7 5700u that gives you 8 cores and 16 threads while only sipping 15 watts of power this means the 49 watt hour battery will get you hours of productivity while on the go you'll also get the AMD Radeon RX and Vega 8 featuring 8 GPU cores that clock up to 1900 megahertz the aura 15 Gen 2 supports up to 64 gigs of dual Channel ddr4 Ram that way you can have as many Chrome tabs open as you need for your productivity with a thickness of less than two centimeters and only weighing in at 1.65 kilograms taking the aura 15 Gen 2 will be a breeze you'll get a 15.6 inch 1080p screen that covers 95 percent of the srgb color space with a peak brightness of 300 nits the aura 15 Gen 2 comes loaded with IO ports including USB 3.2 Gen 2 type c full feature DisplayPort 1.4 a USB 3.2 gen 1 type a two USB 2.0 no type A ports and an HDMI 1.4 Port that includes hdcp 4K UHD at 30 Hertz native also get a gigabit Lan port and a two in one headphone microphone Jack as well as a Micro SD card slot for more information or to configure your own Aura 15 Gen 2 check out the link in the description [Music] your Hardware sorted of course the next thing you want to do is get proxmox installed on each of your individual nodes so of course you'll need to create a bootable USB using whatever means that you like to use to create your bootable USBS and you need to boot from that bootable USB and start the installation process like we've done in previous videos of course the first step will be to set up an email address and a password for your root account next you'll give each of your different nodes a host name an IP address a Gateway and a DNS server so that it can communicate with the internet for updates downloads things like that and then at that point the installation process will proceed and then after several minutes depending on your hardware setup you'll be brought to me login prompt on that screen you'll see the IP address and Port that you'll need to go to in your browser so that you can log into your proxbox server for the first time and of course you're going to do that for each of your individual nodes so if you've got three you're gonna do it three times you've got five five times whatever right so you're going to do this for each of the individual nodes that you're going to put on your cluster okay so once you've got proxmox installed on all three of your nodes what you'll want to do is is get logged into each one of them in separate tabs and then jump over to node number one so here we are on node number one and we're on data center that's where we want to start then what we want to do is come over here to where it says cluster click on create cluster and then we're going to give it a cluster name so I'm just going to be original call it DB cluster like so you've got the option to add additional Network ports here network connections if you will basically the lower the number the the more priority it will have and then any additional network connections that you have there will be for failover in case something goes wrong more bandwidth is needed or something to that effect so once we've got our cluster named and we've got our Network our cluster network setup here what we can do is click on create we're gonna go ahead and give this just a second here and it says task okay so we're going to go ahead and close that right there and then we're going to click here right on that cluster and click on join information here we've got an IP address a fingerprint and some Joint Information what we want to do is click on copy information and then we want to jump over to node two so we're going to come over here again we're going to go to Data Center we're going to go to Cluster then we're going to click on join cluster and paste that in there it automatically fills everything out except for the root password of the first node so we want to go ahead and put that in now like so and then click on join DB cluster so now he says it's stopping the PVE cluster at this point this node has become kind of unresponsive and that's fine that's perfectly normal what we can do though is jump over to uh over back over to our first node and here we can see that node 2 is joining it's got a little question mark right there and here in a moment uh once it has done its thing that will turn into a check mark like it's done there and now we've joined uh the second node to the first node in our DB cluster so let's jump over and do the same thing on the third node so we'll jump over to here we're going to click on uh through cluster join paste put in the root password again that is the root password of the first node not not necessarily the the root password of this node hopefully you're using different root passwords on all of your systems but it's the root password for the first node whichever one started the cluster so once we've got that again we're going to go ahead and click on join the cluster again it's doing the stopping the PVE cluster service we jump back over to here now we can see cluster or DB cluster is zero three uh set up and ready to setting up there and we'll be ready to go here in just a moment so here we are just a moment later and we can see that all three of the nodes are part of the cluster now and we can actually administer each one of these nodes from each of the different dashboards so if we jump over here on DB cluster one or if we come over here to DB cluster uh two just like that and here again we've got a cluster or DB cluster one and DB cluster two and again DB cluster three we may have to log back in there and that's fine so basically my point is you could Minister all three nodes from any of the three nodes and I kind of dig that kind of flexibility in a setup like this so there we go cluster one cluster two cluster three all three of them again access to all three of the nodes on the cluster so now that we have our clusters set up we want to set up storage for each of the three nodes so we can have high availability and replication so let's jump back over to node one and get and start getting our storage set up so here we are back on node number one while we're here though just real quick we can kind of take a look at this dashboard and see that our status is good we've got three nodes online zero offline below that we've got the combined resources of all of the nodes in our clusters we've got 12 CPU cores that's four nodes or four cores per node uh we've got about 23 ish gigs of RAM available of course that's eight gigs per node and then for storage we've got uh 121 Gigs available but that's because we haven't actually configured all of our storage that's just the storage that's available based on our boot drives at this point and on the right side we can see the individual nodes and Below above that we can see our Our Guest Services or whatever you want to call it where we're going to have containers and virtual machines it'll show us what's up what's down that sort of thing so now let's jump over to our first node here we're going to come down to where it says disks and right here we can see we've got two separate disks we have a Dev SDA and Dev sdb the sdb is the USB drive that we're booting from that's got our operating system and the SDA is the one terabyte SSD that we're going to use for our storage now if you if this says usage yes uh GPT yes you may have to go in and wipe the drive in order to use it here um but mine is a fresh drive so we're good to go so what I can do is just come down here and click on ZFS click on create ZFS we're going to give this a name we need to make sure that whatever name we give this particular storage pool the ZFS do we give it the exact same name on all three nodes so that when containers and VMS are moved from one node to another uh their storage is the same the name of the storage is the same so that they can identify where they need to be at any given time so we're just going to call this storage I've got and the reason I'm deliberately naming the storage is because um I plan on making another video where I'm going to take an existing proxmox server that's got multiple containers on it and moving all of those containers over to this three node cluster so if you're interested in that let me know in the comment section hit the Thumbs Up Button uh don't forget to subscribe if you haven't already but basically now we've got our storage name here I'm only using one disk per node however if you had multiple disks you could set up raid this way again I'm just going to use the single disk set up here I want to make sure that ad storage is checked in this little box right there but only on the first node when we get into nodes two and three or or further than that if we wanted to go that far we're only going to have the ad storage check on the first no that's very important so we're gonna come down here we're going to check uh we're gonna check the box next to all of the different storage uh devices that we want to use again I've just got the one so I'm going to click create we're gonna give this just a second to do its thing here there we go now we've got storage set up and right here again we've got that question mark meaning that it is currently setting up in the background so while it's doing that we can jump over to node two and do almost the exact same thing so we're going to come over to uh node two we're going to go to again we're going to check our disk make sure that it is available right here again we've got SDA and sdb we're only interested in SDA here again this one's clean it's ready to go so we can come down and click on ZFS next we're going to click create ZFS again we're going to call this storage again single disk that's what I'm using here I am going to uncheck that ad storage like I mentioned just a moment ago and I'm going to check the box for the device that I want to use for this storage we'll click create and then once that is done uh we're going to go over to uh node number three do the exact same thing discs this looks good ZFS create ZFS storage single desk add storage is off uh SDA is on and click create so at this point we've got proxmox installed on all of our nodes we've created a cluster and we've joined all of our nodes to that cluster and we've got our storage setup for each of the different nodes so the next thing we want to do to get replication and high availability setup is actually create a a virtual machine or a container I'm going to do a container but we need to do that next so we move on to the actual replication and high availability section here so what I'm going to do is I'm just going to come over here to my first node I'm going to go to Shell I'm going to do a pveam update we'll do pveam available and then we're just gonna go find one of these that we're going to use later I always like to download a Debian 11 standard like so we're going to copy that and then we'll scroll back down so now what we're going to do is type in PV eam just like we did before we're going to say download local because we want to use the local storage that's on the actual device and then we're going to put in the name of whichever container that we just download or that we want to download rather so we're going to go ahead and just paste that in like so and I'm just going to hit enter we're gonna give this a second to download uh and save where it needs to save there we go now we have a container available that we can deploy so let's go ahead and do that now what we're going to do is come up to the top right click on create container the node will be on cluster number one we're going to of course change that right here if we want to but I'm just going to put it on one the container ID is fine we're going to call this uh first demo just for the host name it doesn't really matter of course this is just for demonstrative purposes here but now I want to give it a root password and once we've entered that twice we can move on to the next step and click on template we can select the template we just downloaded we can go to disks and for this we can go we're just going to select storage because that's the the storage pool that we just created we want to select a disk size eight gigs is probably fine it depends on what you're doing but I'm going to call it 16 just because for our CPU we're going to give it uh we're gonna give it one core for memory I'm gonna give it 512 Megs mags but I'm going to give it a gig of swap just so I can type in something here for our Network I'm going to come over to here I'm going to make this DHCP for right now for DNS I'm going to put in some DNS like so that way we've got DNS that isn't confined to our local network and then I'm going to click confirm and I'm going to say start after created and then I'll click finish so once this container is up and running then we can move on to replication and high availability okay so here we've got our task okay that means we're good to go so I'm going to go ahead and close this and then right here we can see first demo is up and running just for the sake of it let's do one more just so we've got some more stuff to play with here a few moments later okay so at this point we can see that we have two different containers up and running in node one so now let's go ahead and get replication configured so what we're going to do is click on our first container here and then what we'll do is come over to replication we're going to click add and then we're going to Target one of the other nodes at this point it doesn't really matter which one you Target because we're going to Target all of them through multiple steps here but basically we're saying hey container number 100 we're going to Target cluster node number two and by default it's just going to do this every 15 minutes kind of synchronize the nodes to make sure that everything is current and up to date you can rate limit this if you want to you can leave a comment and have it enabled once you're happy with this click create and then we're going to do this one more time like I mentioned this time though we're going to go over to Cluster number three and we're going to leave all of this the same and click on create then we can come up uh we can click our first option here our first Target and click on schedule now and then we can click our second Target and click schedule now and here I'm on the right where it says next sync is pending we're going to give this amended to do its thing and then it will let us know when it's done and ready to go okay so both of these now say status okay uh how long it took to do it when it's next sync is and how often it's scheduled again 15 minutes so now we've got our first container done now we need to move over and do the second container that we've got here that's number 101 so we're going to follow the same steps we're going to click our container or our virtual machine whichever you happen to be running we'll click add again we're going to do DB cluster 2. we're going to leave everything else here the same because it's fairly standard good stuff we'll click create and then we'll do the same thing here and we're going to come down here and do cluster number three create and then again we can schedule now on both of those and over here again it says pending we're going to go ahead and give that just a minute to do its thing and then once this is done we can move on to high availability okay so now both of these have a status of okay we can say when the next thing is how long it was and again how often it's scheduled now if we want to make sure that that's actually worked what we can do is come over here to our storage on all of these nodes really and we can see a sub volume uh 100 disk zero so volume 101 disk zero if we come over to Cluster number two again same thing cluster number three same thing that tells us that our synchronization our replication has worked so now we can actually move into the high availability part of this so to do that what we're going to do is come over to Data Center we're going to click on h a right here and then what we want to do up at the top it says Quorum okay that means that where everybody's happy with how things are are progressing all of the nodes are working together so what we want to do is add some resources and what I mean by resources rather what they mean by resources are the virtual machines and containers that we want to monitor for high availability so what we'll do is click on ADD we're going to click on first demo here all of this stuff in here you can just leave as it is and we'll click add we'll do the same thing oops right here and we'll do second demo and click add and now that we've got our two resources or containers VMS those are pretty synonymous right now up at the top we can still see our Quorum is okay but now that we can see DB cluster01 is our Master but all three nodes are currently active Okay so now at this point we should be good to go but let's make sure that we are actually good to go by shutting down our first node now before we do that what I want to do is come over here to our first our first demo container here I actually want to go to console and we're going to logged in and I want to get the IP address of that note or that that container that yeah that container and it's going to be 192.168.0.40. so what I'm going to do is I'm going to do um ping oops 192.168.0.40. I want to make sure that it gives me an ipv4 address that's what the dash 4 is for and then Dash T will just keep it going until I tell it to stop so I'm going to go ahead and do that and then what I'm going to do is come over here to my Armada controller we can see that we've got a time of less than one millisecond here so I'm going to come over to my Armada controller and right here I can see prop proxbox node three two and one what I want to do is actually shut off node one so what I'm going to do is come over here to devices I'm going to go to my 24 Port switch I'm going to click on ports I'm going to come over to uh over here oops that's too far and cluster number one is on Port 16. so what I want to do is check or click that box profile is going to be disable and I'm going to click apply then I can come over let's go to node two because node one is going to become non-responsive so let's open this back up and all of a sudden our request has timed out so at this point we can just kind of hang out and wait what I want to do though is uh toggle both of these open and bring this back over so that we can kind of see in real time I'm going to speed ramp this but we can see in real time how these uh containers move from node one to possibly node 2 or node two and three or just three we'll let the system decide what it wants to do here okay so here we are just a couple of minutes later that does take a little bit of time and of course the more nodes you have the longer it may take depending on your network throughput and what kind of a connection you've got those sorts of things you could actually reduce the amount of time the amount of downtime uh when it when there's an outage like this if you had a ceph server set up uh that was kind of a shared storage solution between the three rather than having individual storage nodes or storage pools on each one that are somewhat shared and synchronized but but a a ceph solution would actually be a better way to have a smaller amount of downtime in the case of a node going down so here you can see that it actually did separate them out by going to uh the first demo went to Cluster number two and or the node it says cluster here I really should have named it something else but uh but then a second demo went to uh node number three and of course I can come right back over to here and I can turn that Port back on like so now ultimately that doesn't do anything for us other than just turn the the network Port back on uh it's not going to automatically move those containers or VMS back over to that original node uh it's just now that node is available for if uh more resources are needed by those nodes or another node goes down now that node is back and available uh for use in that cluster again so there you go there's how to set up a basic high availability proxmox cluster using three nodes uh hopefully you found the video helpful if you did do me a favor give the video a thumbs up maybe consider subscribing if you haven't already um also I've been putting this out in my videos here lately if you're interested in supporting the channel and getting something in return which means uh videos with no ads that's no baked in ads that's no YouTube ads just just straight to the content uh you can join my patreon my DV Tech dot fans website or become a member here on YouTube however if you join on patreon or DB tech fans you can get ad-free content for just one dollar a month if you want to go that route so but with that said I do want to go ahead and wrap this up I want to thank you for spending a few minutes of your day with me today and I will talk to you in the next video

Channel: DB Tech

Views: 16,644

Keywords: proxmox, proxmox cluster, proxmox high availability, proxmox ve cluster, high availability cluster, high availability proxmox, high availability, proxmox tutorial, proxmox ve, cluster, proxmox cluster setup step by step, high availability kubernetes cluster, proxmox ha, proxmox cluster zfs, proxmox cluster setup, proxmox cluster 3 nodes, proxmox cluster network, proxmox ve cluster setup, proxmox cluster tutorial, proxmox cluster shared storage, high availability vm

Id: FZnVmt_DvUk

Length: 20min 30sec (1230 seconds)

Published: Thu Nov 10 2022