How to Setup LINSTOR on Proxmox VE

in this video we will be extending a three note proxmox ve cluster with our lens door proxmox plugin this walkthrough references steps outlined in our linstore user's guide as well as our blog links to both will be included in the description below adding shared stores to proxbox is often done in the form of adding a San or some sort of network attached storage it's usually one of the initial steps taken after configuring a new cluster and it's an important step because shared storage enables both live migration and high availability management of VMS and containers linstore effectively turns your proxbox cluster into its own shared nothing sand virtual machines and containers have their corresponding disk images replicated in real time throughout the cluster to other nodes if the proxbox node crashes there's at least one more replica of data for each virtual disk image by Design there is no central point of failure when multiple replicas of data are always on standby to get everything up and running first we need to install the linstore components alongside proxmox then the linstar proxbox plugin is configured to present linstor as a storage option inside proxmox once their storage is configured we'll demo a live migration and failover now made possible by linstore taking a look at our current cluster you can see that we have three nodes proxmox-0 proximox-1 and proximox-2 all residing on the 192.168.223 subnet on the left side you can see that we do not have any containers or VMS running and we only have the default local storage configured for each node in the cluster taking a deeper look at the networking on each node we have eth0 reserved as a management interface eth1 is reserved for lens store and the replication traffic and vmbridge 0 is reserved for proxmox cluster communication and VM and container networking traffic we will be referencing the linstore users guide throughout this video so check out the description for links to get started we need to configure the limit public repositories for proxmox this repository includes all software packages necessary to deploy a lensor cluster and interface with proxmox here we're copying and pasting some commands from the user's guide to fetch and import limbit's public signing key to each node's key ring followed by configuring the limit public repository on each proxmox node once this is done a quick apt update is needed before we can install the required software packages next we need to check which kernel we're currently running and with that information we can install the correct kernel headers now we need those headers because we're going to install the drbd dkms package drbd is the internal component responsible for replicating data between the nodes we're also installing the drbd utilities package and we're also going to be installing the proxmox plugin during this step this might take a little while so we're going to speed up the video now with all these components installed you're actually ready to bolt your proxmox cluster up to an external linstore cluster however in this video we will be installing linstore alongside proxmox on each node each node will both function as a member of the approxbox cluster and a member of a linstore cluster next we'll be installing the linstore components this will consist of the linstore controller package the linstore satellite package and the linstore client package now even though we're installing the lens store controller on all three nodes we're only going to be enabling it on one node proxbox-0 in this case before we enable any Services we need to make a few changes to the linstore satellite service by copying these lines from our user's guide we're going to have the service only report that it's ready once it's had a chance to connect to the linstore controller and enumerate any replicated block devices now we're ready to enable and start the linstore satellite service on all three nodes next we will configure the backing storage for the linstore satellite nodes first we're going to set up a volume group called linstore underscore VG on the dev VDB block device this is basically an extra disk in our test environment here now once this is created we're going to create a thin pool on top of this device and we're going to Grant it 80 of the free space from the dev VDB block device and notice it's simply called thin pool at this point we'll go ahead and switch to a single terminal session now we're ready to start and enable the linstore controller service now we're only going to do this on the proxmox-0 Node as the other nodes are simply linstore satellites once the service is started we can begin configuring our linstore cluster by running lens store node list we can see that we do not have any nodes defined for a cluster and we can go ahead and add nodes by running the linstore node create command followed by the node name and the IP address we're going to be using now we're going to go ahead and pass a parameter called node type of type combined this simply lets linstor know that we may run it as a controller as well as a satellite we'll go ahead and change the values and add the other nodes as well you know you could do this in a for Loop but since we only have a few nodes we'll just go ahead and change the values here if we take another look at our node list you can see that now we have all three nodes type combined added to our linstore cluster now we need to go ahead and Define our storage in lens store before we can use it with proxmox and we can do that with just a few commands and you'll notice that we'll reference the thin pool that we created earlier now we need to go ahead and create a storage pool in each one of these nodes we're going to do that with the linstore storage Dash pool create command and this command has a lot of parameters we're going to break it down after it finishes here okay now that that's been created you'll notice we're passing an lvm thin parameter in there letting linstor know that we're creating an lbm thin storage pool then we're passing the node name and we're calling this storage pool PVE storage and then the last parameter is the volume group and the thin pool that we created earlier in the video we need to go ahead and also create this storage pool on proxmox-1 as well as proxmox-2 now we have the same storage pool defined on all three nodes we can view this by running the linstor storage Dash pool list command here you can see we have a storage pool named pve-storage on all three nodes using the lvm thin driver type referencing the volume group and thin pool that we created earlier next we need to define a resource Group that functions as a template that gets applied every time proxmox requests storage from linstore you can see from the command above We've ran linstore resource Dash group create and we've passed in pve-rg as the resource Group name and then a storage pool parameter targeting the PVE Storage storage pool that we created earlier and then finally a place count parameter of three this place count parameter simply means we're going to have a replica of data on each node in the cluster now as a rule every resource Group needs a corresponding volume group created and we'll go ahead and do that now with the linstore volume Dash group create command now to double check some of the commands we just ran we'll go ahead and run a linstore resource Dash group list command you can see that we have the default Resource Group that gets Auto created when linstor is installed and then we have the pve-rg resource Group that we just created and we're going to be using to interface with proxmox now we're ready to configure proxmox to use the linstore proxmox plugin to do this we actually have to edit slash Etsy slash PVE slash storage.cfg this file is replicated by proxmox to all nodes in the cluster so we only need to edit it on one node now I've already gone ahead and defined my storage for linstore here you can see that begins with the drbd storage type and I've called it linstor underscore storage and for the content we're only focusing on the images for the virtual machines and the root directories for the containers we've referenced our controller with our controller's IP address and then we've referenced the resource Group pve-rg that we created earlier after making changes to this file we'll need to restart a few proxmox Services before we can request storage from linstore I'm going to copy and paste this command in here to restart five proxmox Services pve-cluster PVE demon PVE stat D PVE proxy and pve-ha-lrm now once this command finishes that'll Force proxbox to re-enumerate the storage in the system and it will recognize the lens or storage that we just configured now at this point we can go to our proxmox GUI and we can see that all three nodes have the linstore storage added on the side here navigating to the storage configuration in the GUI we can see that the storage we've defined in the storage.cfg file is shown here as well everything's good to go so let's quickly set up a VM and then we can demonstrate failover as well as making a VM highly available now we'll quickly set up an Ubuntu VM using linstor's storage we'll call it Ubuntu test we have some previously downloaded ISO images here and we'll shrink the disk size down just a little bit and notice we're selecting the lens door storage that we defined earlier up the core count memory is fine networking's fine and we'll go ahead and start it after we create it now behind the scenes here it's actually provisioning storage and lens store syncing that storage between all three nodes and then it's starting our VM and we'll Breeze through this and we'll be back okay we have our VM running and configured so we'll go ahead and open a shell session and we can see that we have a Hello World text file currently open in VI now if we run our linstore resource list command we can see that we have an active resource on proxbox-0 for this vm's disk image now we can migrate this virtual machine between nodes almost instantaneously the replicated disk image simply changes from being primary on one node to another node and our hello world session is still running except our VM is now running on proxmox-1 coming back to the command line if we run this command one more time we can see that linstore has moved the storage from being primary or in use from proximox-0 to proximox-1 and for a final demonstration here we're going to take this virtual machine and we're going to let proxmox manage it as a highly available virtual machine so coming down to the ha configuration in the data center we can see that we currently don't have any resources defined so we'll go ahead and select our existing VM and we'll just leave a quick comment here notice our request state is started we don't need a group and once we add this here we'll see a local resource manager start and proximoxnow recognizes this node is active now behind the scenes here I'm going to force power off proxmox-1 to simulate a node failure now this next part is going to be sped up a little bit because it can take a while for the remaining nodes to determine that proximox-1 is in fact down and that our Ubuntu VM can be started elsewhere okay our VM has been migrated and now it's currently starting the GUI might lag a few seconds behind what's actually happening in the cluster now it shows that it started we can go ahead and open our shell session here and we can see that our VM is running it's booting up right now our Ubuntu virtual machine since we've made it highly available can now withstand a virtual host failure to recap this video we started off with the proxmox cluster that was installed using the PVE no subscription repository then we added the public limbit repositories for proxmox we installed all the software for a functional linstore cluster and then we configured lens store configured the lensor proxmox plugin and then demoed a live migration an AJ operation are now possible and again we did this all for free using the free public repositories out there if you'd like some help if you want to learn more if you're interested in Enterprise support for your proxmox cluster anything like that feel free to reach out to us again our website is jam-packed with tech guides users guides blog posts Etc there'll be links in the description and as always I hope you've enjoyed this video and thanks for watching

Channel: LINBIT

Views: 2,750

Keywords:

Id: pP7nS_rmhmE

Length: 15min 7sec (907 seconds)

Published: Mon Jul 24 2023