Your Complete Open Source Virtualization Guide For Getting Started With XCP-ng & Xen Orchestra

in this video I'm going to walk you through getting started with xcp and and Zen Orchestra we're going to start with loading the hypervisor and take you all the way through creating your first virtual machine I have several videos I'll be referencing along the way that offer deeper tutorials on certain topics and you will find those right in the description down below I have time stamps on this video for those of you that just want to skip ahead so let's get [Music] started are you an individual or Forward Thinking company looking for expert assistance with network engineering storage or virtualization projects perhaps you're an internal it team seeking help to proactively manage monitor or secure your systems we offer comprehensive Consulting Services tailored to meet your specific project needs whether you require fully managed or co-managed IT services our experienced team is ready to step in and help we specialize in supporting businesses that need it Administration or it team seeking an extra extra layer of support to enhance their operations to learn more about any of our services head over to our website and fill out the hire us form at Lauren systems.com let us start crafting the perfect it solution for you if you want to show some extra love for our Channel check out our swag store and affiliate links down below that will lead you to discounts and deals for products and services we discuss on this channel with the ad read out of the way let's get you back to the content that you really came here for now we're going to start here by talking about the hardware because the first question is what what do I load this on what does it support and xcp if you would have followed some older post or maybe you found something from three or four years ago when they first released it and said hey it doesn't have support for like Intel 2 and A2 gig Nix or it doesn't work well with certain ryzen or epic processors those are issues that have been fixed they have updated the kernel packed in the new drivers the production release here in February of 2024 is 8.2.1 that does have the extra drivers in there we're going to be using the 8.3 which is in beta 2 so I'm going to say it's pretty close to being released that's what we're going to use for our demo here which also has updated drivers and by the time you're watching this will probably be in full release but if you just want to stick with the 8.21 yes it does have that Intel 2 and half gig support and better support for ryzen and epic or overall AMD processors matter of fact I have a video of a ryzen system I built I'll leave link down below that has been running for the last few years my xcp production systems and lab systems kind of both that we have at my office office now as far as doing things like rate arrays that comes into question quite a bit and that's why I pointed out that this does have the ability to put two drives in it and we can set this up as a raid array but as of right now in February of 2024 there is no UI to manage that raid array it is simply a Linux raid managed system and that includes if you choose ZFS yes ZFS is supported yes you can choose that as your storage device on an xcp system but it is manually set up and manually manag you manage the storage repositories through the system in terms of once you've created them but the underlying like looking at snapshots and things for the ZFS specifically not snapshots for the M that is not handled within the UI that is all handled from the command line and the same goes if you have any issues with your rate array you would be rebuilding it from the command line I just want to put that here for some clarification now if you're choosing some Enterprise Hardware to run this on frequently let's say dell has a built-in controller that presents the drives as a single drive because Dell is handling the raer rate well then this won't be an issue at all and of course I've always talked a lot about on this channel using an external storage server as your target for your system such as either a sonology or a trath so you can also use those external storages but once again they're presenting the storage and just bringing it to xcp but now let's get into actually loading xcp cuz that part is pretty simple and yes we're just going to use local storage to keep this whole demo really easy to follow now if you're looking to do a deep dive into the storage systems that drive xcp and how migrations work how coalescence works you'll find that video linked down below it's a deep dive into that topic and all the different features around it now once you've determined which Hardware you want this to run on the next step is to download xtng from xcp.org they give you some pretty basic instructions of how to copy it to a USB drive there's also an option on Windows that you can use ruus this does work with ventoy as well now we're going to do the 8.3 and I'll leave a link to this blog post to talk about the beta 2 that was released in February 15th of 2024 and by the time you're watching this might be out of beta and in full release there is a link down here to download it the boot installer is pretty straightforward the first thing it's going to ask is which keyboard layout we're going to choose us for mine it checks for any existing products if you're going to be doing an upgrade finding none it'll just go through and let you accept the license agreement and go forward now as I mentioned earlier you can do software raid so we'll go ahead and do that for this particular setup to show you how it works simply choose the drives go ahead and hit create lets me know it's going to rase and it's going to create the array for me and it's going to be a simple mirror with that being created we're going to click okay now from here it's asking where you would like to do the virtual machine storage it did not need all of that space for the install of XC PNG so the leftover space we're going to set up as an EXT volume and that will allow thin provisioning it does give you the option for lvm if you'd like to use that I recommend EXT for standard then provisioning and we're going to ahead and click okay please select a type of media local media is fine unless you have some extras I like to live dangerously so we're going to go ahead and skip the verification and from here we want to set the password you can change this later so I'm just going to set it to something simple we're going to ahead and ipv4 that's fine this is where you can choose automatic via DCP or set your static IP I'm fine with DHCP so I'm going to go ahead and let it do that if you have a management VLAN that you'd prefer this to be on there's an option here to choose it all these things can be configured later we'll give it a name and we'll go ahead and let it automatically configure the dhap servers now we're going to choose a time zone use DHCP for ntp servers use default ntp servers or provide them manually these are all great options but make sure if you choose dhtp that you do have it set up to provide ntp service in your dhtp if not you can use the ones that are the defaults and we're going to go ahead and click install now the install is complete we're going to press okay and let it reboot now once the system boots up it gives us the IP address to the system the network management interface and if we needed to make changes this is how we would do this we can configure the management interface again rout go ahead and put that password you set and then from there we could go through and change these disabling it or choosing a different one we're going to ahead and back out of this because we don't need to change any of that but this gives us just basic functionality of the host it does not have a lot of features in here it will start and stop any VMS running on this host you can show them currently because we just loaded it there's none and if you went to this IP address you would not be greeted with a normal management interface like you might expect that is all handled through Zen Orchestra so actually the next step is we're going to load Zen Orchestra on this now if you do go to the IP address of the system after you first load it and type in the password you will be greeted with EXO light now EXO light is still very much under construction it is a work in progress that there'll be a lot more progress made later in 2024 but for now it's kind of well under construction like it says they're making a lot of Headway you can follow along in the forums on this it will do basic tasks in terms of starting and stopping virtual machines but it doesn't have a lot of other control beyond that in the future though this will be a default load when you load X cpng you'll be able to get to exolite connected to your system and perform quite a few of the functions in it by the way this new UI design that you see for exolite is going to to be similar to the new UI that is upcoming in the newer versions version six of Zen Orchestra we're going to keep with the current version The Five series but in the future this is also going to be a common theme that both of these will be following they have an entire Dev blog that really explains their concept for UI they're doing this very carefully so this is why it's not being done with rapid deployment from the time they announce but they're doing it in a very concise way so there'll be a very common theme between them and a very good UI that they're coming up with but for now we're going to keep keep using the Zen Orchestra 5 series and start from there now let's talk about getting Zen Orchestra deployed the challenge you may find if I watch my build from sources video which you'll link down below is that if you want to build from the sources and you want to run it on the VM how do you get that initial VM setup on there well there's an interim way to do this which would be load Zen Orchestra from sources using something like virtual box something I mentioned in that particular video that you can load it on a separate system Zen Orchestra does not have to run on the host itself it can run on a separate system or even another host you already have this is actually kind of an interesting relationship Zen Orchestra has with xpg the hypervisor host is it's a on to many relationship one instance of Zen Orchestra can run many many instances of the different X cpng hosts I have a whole explainer video where I talk about how the architecture of the Zen API works and like I said you'll find that link down below common theme you might be hearing here that being said how do you get it on the host itself and what is is there a easy way to do this or what if you're a business that would like to use the full version completely supported from the team over at Vates well they have a really simple deploy meod for the Zen Orchestra Appliance so Exo refers to Zen Orchestra xoa refers to Zen Orchestra Appliance the appliance is the completely auto updating Service delivery model that you can get a license for they have all of the features but you get quite a bit of functionality without a license matter of fact you can even get a free 30day trial as of right now they've been offering that for a number of years that will give you all of the features if you want to test them out but even with the basic deployment of Zen Orchestra it's pretty easy to do now there's two ways to get the Zen Orchestra Appliance deployed you can do from the command line SSH into the Zen host run this one liner right here you'll find a link to that in the description and from there it will download the Zen Orchestra after you've answered a couple questions about whether or not you want a DCP the IP address Etc and you'll be able to log in option two is doing it through their web interface you can actually go to the VAP site and they have the option where you can actually connect from their website to your local server and it'll build a connection that allows the Zen Orchestra Appliance VM to be imported directly in there and get going no matter which option you choose whether it was the xoa appliance or building from sources The Next Step I want to do is connecting your Zen Orchestra instance to your host because even if you Auto deploy it it won't autoconnect to that host you do have to put the credentials in and then from there I will show you how to set up the networking and then we're going to build our first VM now this is a Zen Orchestra built from sources that I'm will be using for this if we click on no support it lets me know that one it's up to date it'll see the commit version I'm running at this is the latest version available as of February 22nd 2024 but when we click on home we'll notice it says add server because there's no servers in the list here you can click on the add server or go to settings and servers now as I noted you can have multiple servers in here we're only going to need to connect one one for this demo but we're going to type in XP n and give it whatever name we want so we'll call this one my YouTube demo the address now when I installed yes for those of you that were sharp eyed and watching 17216 16195 was the address but I actually move that to a reserved address of 49 put the username in put the password in we're going to check the unauthorized certificate and we're going to ahead and hit connect and you can see it's connected it lets me know cuz it's green here and we have a pool called xcp test now we can rename the pool right now if we want but I want to point out that the name of the pool is the same as the name that it was assigned when we set it up so if we go here to the home and then we choose host so we can see the host itself there's only one host in here this is called test we'll talk about the patches later and we can rename the server as well so we can call this our test server that's perfectly fine but let's give the pool a better name pool pools are the resource pools is what I'm referring to I have a whole video where I dive deep into the functionality of Zen as I mentioned earlier and it explains what resource pools are but they're essentially pools that are of one or more machines all grouped together so what you set in the pool applies to all the machines this what we're going to call our YouTube demo and if we want to make some notes Here we can just click and type them this is a common theme you'll find throughout Zen orchestra that there's a lot of spots that you can just simp click and rename things now while I'm under the pools here I want to go to the network and change the network settings to make this a little bit more clear I've only got one ethernet plugged in for this and we're just going to go ahead and rename it to e zero but by default it is called pool wide Network associated with E I think e is easier I'm not using this one at all so I'll just call this one not in use that way when I assign this to a VM is pretty obvious this is not one I'm plugged in and this one is e z I could give it a description if I wanted to to what network it's connected to it's kind of a personal preference just for whatever makes sense to you now while we're here on the networking I'm going to show you how to add a VLAN so we're going to go here to add Network and I want you remember you always add them from the pool side and then all the hosts within this pool will have the same uniform network configuration and this is just going to be VLAN 10 for our demo here give it whatever name I want hit the VLAN in and hit create network now we have VLAN 10 created just so we have one more option when we're setting up for our VMS now the next step is to add a new repository to store isos so we can eventually get to loading our virtual machine so we want to do a new storage and select the host well there's only one host in here to select storage name we'll call this one ISO I'll give it the same description we want to choose that it is a local ISO storage and we're just going to use slash media this will create the folder on the system directly for us to upload it to and we're going toe hit create now that we have the iso created we want to upload an ISO to this so we're going to go over here to import and we want to import a dis to import an ISO into an ice repy and ISO repository is required we've already done that this actually just links to the instructions to do it and from here we're going to select our ISO and we can drag and drop one in here or just click it and choose to upload or they has the option to download it from a URL and give it a name so we can pull in that way I already have one downloaded so I'm going to choose to grab it from here I'll choose my Debian 12 ISO and I just click import and it's going to upload it right into the repository all right now that we have that loaded I can add a description if I want that's completely optional and I can delete it if I want right from here or even rename it and give it some other name if I want it I can just maybe call it de Debian 12 but I like leaving the full name because I do know it's a Debian 12 and it is the net install version so I'll leave it at that now from here let's go ahead and create a new virtual machine we're going to go to new we'll select the pool that we're connected to we'll select a template now these templates are just to set some general parameters it is not required that I have a Debian 12 template so I'll choose a Debian 11 one but it can be renamed like Dean 12 just by typing that Dean 12 how many CPUs we're going to go ahead and go four CPUs sounds reasonable we'll give it 2 gigs of RAM pxe boots an option if you have that set up but we're going to go here and we're going to choose our wn2 ISO once you have a VM loaded you can switch it out for the guest tools to load the different tools but for now we'll just choose this we'll leave this fine this is our e hero and as you notice there's our VLAN 10 and not in use now it's nice and clear from labeling it and I'm just going to go ahead and let it autogenerate the MAC address but you can specify it and you if you want to now or this can be done later add more than one interface it will let you do that right here as well go ahead and leave it at one interface then we can choose the storage we'll just let's call this de2 we can leave it at the local storage repository it's the only one we have and we'll just let it say created by Exo and it we maybe give this 60 gigs of space if we wanted to add more discs we can simply do that as well and we'll go a and close that and close that if you go to advanced settings here whether or not you wanted to boot VM after creation auto power on or use cloud andet it has some more advanced options including repeating so you can actually build these if you wanted to build quite a few of them at a time and you can come up with a naming scheme up but we're only going to create one for Simplicity here and we also can choose the default bios or choose ufi I'm not worried about either of those we're just going to leave it at bios and go ahead and hit create now from here it's started up the VM and we're going to go to our console and run through a Debian install to make this a little bit more viewable I'm clicking this to show it brings us to the screen we'll go ahead and set up dbn really basic here if if you leave the root blank it will automatically allow you to create another user and create Pudu so let's go ahead and enter that now if the systems recalled it'll automatically reboot and now we have a work in Debian 12 VM installed one thing I will note here is the drive is still installed we can just click this eject and it will take out the dis and if we go under here for advanced and we can see the boot order is hard drive DVD Network and this is as it should be but if ever we needed to move it you just drag this up and hit save if we wanted to boot from DVD again to do a reinstall it's pretty simple to do it just may not be obvious that you just drag and drop these and click save now if we go over the network tab it's not showing the IP address of the system and as is because we don't have any of the guest tools installed and if you go over to General you'll see management agent not detected so let's go here and to the council and log in and there's two options we can do from here we can select the guest tools and attach them to this one Mount them run the guest utilities on there that would show the IP address and load the management tools or they do have some more updated tools that you can run that are in currently beta and I'll leave a link to those down below it is their new tools written in Rust but for Simplicity we'll just use the guest tools to load the guest Utilities in Debian we're going to go sudu Mount Dev CD RAM 2/ mount cdrom I already have a directory called that put the sudu password in go to my SSH Mount C ROM Linux and go ahead and just run the install sh don't forget to add sudu if you're not running as root say yes now the utilities have been selected and run it's installed and we can go over here go to the networking and we see the IP address that's been assigned to this and if we go over here to General it says management agent 7.3 detected this also allows for the dynamic changing of memory so you can see that the memory is now being detected in the stats here now while we're on this stats page let's go ahead and just stop the virtual machine hit okay and I want to point out that each of these are clickable and so we can change this to eight cores for example and we can change this to 4 gigs of RAM and we can change this right here if we need a bigger drive and just click the drive and maybe we want to make this 80 gigs now making this 80 gigs only changes it here you still have to go into the OS itself and expand the partition unless you have something set up to Auto expand it but this does easily set the size difference to now 80 gigs and if we go ahead and restart this VM we are going to get a no hosts available and I wanted to point that out because if we go over here to our host and we go and look that we only have four cores available on this particular host that's why it will not let me start this sometimes is where people can get a little bit confused so we'll go back over to our VM and we'll go to advanc if you read the logs it's telling you there's no host with that many processors available and I can't just downsize it to the four cores well I can but it's still going to give the same error because once you've upsized it to a certain size you have to go here because you have the limits set we have four now but we want a maximum available of four so let me explain how this works if we set the first one to sing Le core this means we're going to start with one core but even live can dynamically upgrade it to four cores I want to go ahead and start it let's say with two cores but labor I want to expand it but I don't want to reboot it and the same thing goes for the memory we have the minimum and the maximum that you can set for the memory it's four and four but we can also set it to 4 and 8 so it's still going to have if we go over here to General 2 and 8 gigs of RAM let's go ahead and start this VM up and once it boots and a management engine is is detected we can now go over here to General and change how many cores it has we can up it to four press Center and now without rebooting this VM we've changed it to update the number of cores and then we can also shrink or grow the amount of memory depending on how much is available so now we're going to shrink this VM to only 4 gigs of memory I've got a whole video about provisioning memory and doing this dynamically and some of the challenges that can come with applications running while it happens some applications are fine with it others may not be but the OS itself Debian and windows both can actually have their memory changed while it's running without crashing now if we want to change the network settings and we did create that other VLAN and we can go here to networking and we can choose VLAN 10 the other one in there and once you have the management tools you can dynamically change these on the fly without restarting the VM matter of fact while the VM is running we can go back over to our pool we can go to the demo here and we can create another network if we wanted to change this network even while it's attached so we can just call this one NSFW and then we go back over to RBM and we'll see that the name is now changed as well these Chang dynamically and it's quite convenient because now I didn't have to do anything to actually swap the network around to make that work now the next thing I want to point out are snapshots before I do that let's go over here to the console and log in I'm going to install the package btop so I have something running on the screen here so I'll leave this running and we're going to go over here to snapshots and instead of just doing a new snapshot which is easy I could simply do this this is a snapshot in time of this particular system but let's go ahead and do a new snapshot with memory and when we do this it takes a little bit longer but I want to show you the state that it brings it into because you can do more than just snapshots and XC PNG you can do freeze frames of where the memory was in all running processes so this pauses a little bit here and now we have a snapshot with memory you can see how it says memory saved it's got the little green versus the normal snapshot here we can go to the console here and see everything running and let's go ahead and quit and just run regular top so I'm still doing something on the machine I've got it running top right now so I should be able to go back to the snapshot here and hit restore so we want to revert VM to the snapshot I don't want a snapshot before of it currently running let's just hit okay and it's going to take a second and it's going to restore the VM go back to the council here it's taking a second to push the memory this machine is not particularly performance oriented that we're running this on this is just a little mini PC and there we go it's exactly where we left off from a frozen State essentially of it with the memory restored to all the applications running as we left it before including I had ejected the disc and it brought the disc back the other thing to note about snapshots in Zen so go ahead and Purge these ones here Purge this snapshot when you do the snapshots it's more than just the drive itself you're snapshotting so right now we have this with 4 gigs and four cores and we want to do a snapshot of it I'm not going to bother with the memory one again but now let's change it let's go ahead and reduce this to a two core system and then let's go ahead and reduce the memory down to 3 gigs so we've now made two separate changes on here and we can even make some notes test hit okay and we'll call this dean2 put some more words here now let's go ahead and revert this back now we didn't do a memory one so when we do this reversion it's going to go here to revert to snapshots and we're not going to snapchot before and it's going to reboot this because it's not a memory snapshot so it's going to bring it all the way back and revert it and start it up but you notice it fixed all the things we did the notes are gone the changes here back to four and four everything is as it was when we did the snapshot these are great so when you make these modifications it's bringing not just the settings of the drive or certain settings it's bringing all settings related to this VM within that snapshot back to the point at which you made the snapshot and from here if we want to go ahead and delete the snapshot I don't really need it I can just go ahead and remove this snapshot hit okay and now it's back to having no snapshots but I want to point out a few more things we can do a snapshot of this virtual machine and then from here we can create a VM from this snapshot by clicking this so if we do this name copy we'll just call it deian from snapshot full copy or fast clone I'll go ahead and hit fast clone fast clone means base it off of the other one which is going to be pretty much instantaneously because they share a common base but not really a big deal when they share a common base in X cpng because this did not tie this particular VM to the other VM it is forked from it but if I wanted to move this virtual machine to another server another repository it will automatically do the work in the background to split it from it so we can start this one up here and I can show that even though we have this one up and running that we fast clone from that snapshot when we get back over to our VMS I can now take this VM I can go ahead and Purge the snapshot I can matter of fact stop this VM it's now off and we can completely remove it and in the background it's automatically doing the work that needs to be done to take that fast clone and all merge it into this particular clone and we can see that by going to disc clicking on local storage and you'll see where it's coalescing we have two pieces to coales it is a two pieces from the original VM and the snapshot now they've coess into the new VM that we created XC PNG handles this seamlessly on the back end it's actually a really nice feature that the system has we can go back over here to the VM and let's rename it just call it Debian 12 again it's not really from a snapshot anymore and once again we can start the process over create more snapshots or we're going to go ahead and stop this VM we can also just fast clone a VM once it's in a stop state so we can click this a few times go back over to home and now we have several clones that we can then group start up which I will probably get an out of memory here I don't think there's enough ramist to start all of them at once and it will give you a warning are you sure you want to start five virtual machines and it was only able to start three because it ran out of memory for the other two we'll also just go ahead and remove these other two go ahead and hit remove you do have to type the words out Delete to BMS and now they're gone when you're in this view right here you can also expand these out so you can see statistics on each of them this is just another view that you have and yes it's going to start showing the IP addresses as these boot up and then we can click on each one of these and look at the actual running virtual machine the next thing want to mention for management is tagging so if we go here and we'll call this one Tom maybe you want to call this one production as well because it's a production VM and now we can put these tags on here these this is the only VM that has a tag so we can see the tags once again by expanding it out like this we can see the production or Tom we can also go in and add it to other VMS we can say let's go ahead and add Tom to this one this allows me to then type in VMS tagged with Tom that's these two here but not that one the Clone ones I added that tag as you can see here but I did not have that tag on that one same with clicking production only shows this one and we can have our tag production this is a really nice feature that is throughout the entire system through Zen orchestra that makes managing things a lot easier you can add tags to each of the hosts in here we only have one but we'll call this our production host and this helps us distinguish when we have more than one host and for example let's go ahead and go in here to our servers label another host hit connect and now we have two hosts connction to this matter of fact if we go here and we go to our hosts you can see we have the test and one called xtng server one I actually used this one in another demo that I had where I was showing how to set up networking and this one's called YouTube demo pool if we look at the pools one is called XC PNG YouTube demo and this one's called YouTube demo pool this one's letting me know that there's a bunch of missing patches for host in this one this one's fully patched and up to date so there's no patches on this notice here we'll get to loading into patches in just a moment because I want to cover just a couple more things related to the tag this is something that can be very confusing when you look at the storage because we have more than one thing called local storage local storage you can see on the end One belongs to XC PNG server one and one belongs to XC PNG test now you can rename the storage and this one belongs to local test storage and we'll just go ahead and rename it but we can also tag them this way where we set things to be production then when we look at the storage go back go back over here to storage and expand these out we can see that this is a production storage and we can set filters here only show me production storage and filter them out so when you have a lot of hosts connected across multiple resource pools you can filter things rather quickly throughout the filter at the top here and as I said this applies to VMS hosts pools and so on and so forth now let's touch on loading the patches cuz there's more than one way to do it we can simply load the patches from here it will do if there's multiple host in a pool a rolling pool update or you can install pool Patches by clicking here something else you can do you can go to the host we're going to go into our xtp and Z test and we want to go to the Council on this one you could also just ssh in and from here you can you just type yum update and it will grab all the packages and you can say yes and it will install all the latest packages on this system pretty simple to do so either way you want to do it whether you SSH and do update and say yes I like it because you can watch them or you simply click the install pool updates the updates will get loaded and installed from here now this this is loaded we'll go ahead and hit reboot and we're going to hit okay here some VMS cannot be migrated without first rebooting host please try and force reboot it will not let you reboot until you stop those VMS I wanted to point that out so let's go over here and find the VMS related to that pool so we're going to filter for that pool the xtng pool we're just going ahead and select these we're going to hit stop and it we'll go ahead and shut these VMS down then go back over to the host XP and test and let's go ahead and reboot this and now the host is fully updated we can go over to Patches and it tells us our host is up to date now let's talk about how VM migration works we can go back over here and we have all these different VMS and because if we went over here to settings and servers we added two different servers our demo one and the another host that we added you can actually migrate VMS between different resource pools so if we look at the two different pools we have the XC PNG YouTube demo and we have the YouTube demo pool we're going to go here to our VMS and we can even do this live it doesn't have to be a off migration we can fire this one up right here all right now that you can see that this is up and running live I'm going to go ahead and log into it so it's got something to do now we're going to go ahead and click migrate there's only one host so it's the only one in the option here but if you had other host and you notice that this is on a different pool cuz it doesn't care it doesn't have to be part of the same resource pool to do a live migration and it will figure out that we called e zero the same on each of these but we could line up different networks if there were more available these are the only ones that sees and then what is the network on the destination where I'm going to use EO because I have that set up on that second host as well so we'll just leave this all the same and we're going to click okay and this will actually kick off a live migration between two completely different hosts off of two different resource pools this will take a few minutes cuz these are as I mentioned earlier not super high performance systems but it will do live migrations or non-live migrations between Any Two Hosts provided Z Orchestra is able to talk to both of them and provided there's a network connection that both of these are able to talk over to negotiate and move the data across all right and now that VM has been moved and now it's on the other server that's called xcp server one and you can see it's still running and working perfectly fine now let's take the time to talk about resource pools now resource pools are called clusters in other hypervisor platforms and they're very similar they're way we can take multiple hosts and put them into a single pool to make things easier such as shared storage or as we Define networks Always by the pool even when there's only one host in there this will actually allow you to define the pool Network and when you add a network it'll add it to all the hosts in that resource pool now resource pools are very similar to what other hypervisor platforms call clusters they work in much the same way with Zen server kind of being an exception the fact that it has no problem migrating a VM between unlike resource pools provided as I said the processor the same you can even do it live but even if they're not the same hey if you shut down the VM you can migrate them without combining them which may ask do I even need a resource pool and in some cases you do not but if you plan to do things like high availability or one use a shared storage between multiple hosts you're going to want to combine them in the resource pool now this represents how things are set up right now we have two hosts and each one even though they're only one they technically are in a pool and they're both being able to talk to this one instance of Zen Orchestra The Next Step we're going to do is combine these hosts into a single resource pool so by combining them into a pool they're still going to have their individual local storage but they're now going to have a common network setup and they're going to have a common setup when we add things like shared storage we can add shared storage once and that one time shared storage add as it to both systems provided we've set the networking up so the each one of the hosts can talk to that shared storage and this also makes VM transfers really easy now if we want to go full ha that would require a minimum of three hosts we're not going to do that in this particular video but I just want to give you the idea of how you would add these now one thing that is really important here is we have to pick which host we're going to join to the other cuz they're always in a pool it's a matter of someone's going to have to lose their pool in order to do that a prerequisite has to be met is we have to take and make sure there's no virtual machines and no configuration on the incoming one so we're going to pick which host is going to be the pool we bring into and then the other one we're going to delete all the virtual machines and make sure there's nothing on there generally when you're adding these to a resource pool you are going to have one that's already set up and you're going to then load bring it into that resarch pool so usually deleting VMS isn't something that's needed by the way though if you have two separate systems and you decide later you want to combine them as I noted earlier without them being in the same resource pool you can still migrate all those VMS you could just migrate them off of the pool into the pool you're planning on bringing them into from a planning standpoint that works perfectly fine now one more thing I'll comment on our resource pools is there's always has to be a master now the master is where the source of Truth is for all the metadata that is related to all the VMS the network settings Etc but that data is replicated onto all of the other ones in that same resource pool so you have one set to master but then all of them get a replicated copy of the data and in an ha setup the master if it were to fail it will automatically choose a new master within the pool as I mentioned though that requires three or more so what happens in a situation when you only have two or even 10 in there and there's no set a ha Quorum for them well you can still force a host that was not master to become Master because it has all the data it needs it just doesn't have the rights to be master and there is a way you can go into any of them in an emergency situation if you were to lose that Master host you do this by going to the council and set new master and it will Echo out to the other ones and you will be able to still manage that particular resource pool this is something else I want to note about the way Zen Orchestra works and you'll see this when we do this we have two servers under the server settings once we combine these you're only going to see one because the Zen API only ever takes its cues or talks to the master and that data is as I said echoed across all the other ones now there are some limitations to having a resource pool as I said you can have as few as one but the maximum really is 64 once you go over 64 I believe you can set it to maybe work with more than 64 host but that's when things can uh get outside in the recommendations so no more than 64 is the general rule here in February of 202 for for any resource pool now we have our two hosters set up under servers and we're going to choose YouTube demo pool as the incoming pool and xcp NG demo that we've been working with as the one we're going to merge into this one here first step is going to the hosts and I want to make sure that I see that they're on the same version so XC PNG server one this is xcp test and we'll just rename it to so we have some consistency this is the one we're going to be bringing into the other system I want to make sure that all the patches are loaded that's on the same version so this is version 8.3 we go to the other Host this is version 8.3 and as I noted both have all the patches loaded the next thing you want to do is make sure that there are no VMS so if we choose the pool and we look at the demo pool here we don't see any VMS I've already deleted all of them and we should also note that if I look in the pool look at the demo and look at the networks I should have no extra vlans or any other network settings other than the base config of eth1 e zero just the two default networks I'm not worried about the names that have been given that are going to assume the names of the incoming pool so let's go over to the pool go to the YouTu demo pool and let's go ahead and add a host it's the only host in here so only one shows up in pull down and we're just going to click okay and now we have the pools combined it lets you know right at the bottom here which one is Master changing which one's Master is actually really easy we can go here to advance and you just hit the pull down before I do that I want to show something if we go over here to our VMS I have a VM running if we can go to the console and you can see it running here and we have this as the master server as noted when we go over to pools go to the YouTube demo pool it says it's Master here but if we wanted to switch which one's Master not a big deal we can say server one or we can switch it to be this xcp and2 now let's look at something before we do this we go to here to settings and then servers you only see one server even though we have two different systems in here that is because Zen Orchestra only ever talks to whichever server is the master if you were to try to add another server in here that was already part of a resource pool that was joined it would give an error that it's already joined in part of a resource pool so it's always going to only have one server listed in here for each resource pool that you're attaching to so when we bring in another resource pool for example we only need one of the hosts that are in that resource pool let's go back over here though and and show the VM running that I have right here and show what happens when we do a switch so if we go to the pool itself go to Advanced and switch from server one to server two it says this operation may take several minutes do you want to continue this is fine it will take several minutes but we will lose access momentarily to Zen Orchestra but the VMS themselves are going to continue in the state that they left off then so it's preparing in the background to do this and all of a sudden it goes blank once it goes blank you just have to sit patiently and wait and after about 30 seconds maybe 60 seconds depending on the speed of the systems on there you will see this automatically reconnect and you'll be exactly where you left off all the virtual machines have continued to run everything in the background was functioning fine you just lose access to make any changes because it locks them all while it resynchronizes and chooses the new master all right now the systems come back and if we go and look at the YouTube demo pool go to advance we can see that this server is now the master so something of note if we go over here to the hosts and just for consistency this one's called xcp server one and this one's called xcbg 2 let's rename it renaming it will not break anything so we can call this one server 2 just so we have some consistencies because it's all being communicated by IP changing the name of this server is not going to break or change anything it's Dynamic and can easily be renamed on the fly like this now let's talk about the networking really quick here we have our YouTube demo and we have the network whatever we Define here as I have this network defined it will automatically propagate to all of the hosts in this pool one of the advantages of the resource pool that we have this one called not in use I can just change it to do not use and now if we go to our hosts we can see under networking it says do not use this is also going to be in the other one too so if we go back same thing says do not use also each host still maintains their own IP address as they were set I have them set via DHCP but if you want to adjust the IP address of any particular host you can even set it to static right here or go into the EXO console by sshing in and override and change which one is set to management we'll cancel out of this and we can look at the other server Network and we can see that this one right here same thing I set it DHCP but if we wanted to set it to static or set a different IP address you still set those individually also as you add networks like this one has this VLAN 10 you can assign static addresses if you need something for the host to be in that Network it's not always needed but under certain use cases for example if you're adding a storage Network that would be where you would set it up is you would add the network through the pool and then you would set the IP addresses to these servers now let's talk about storage a lot of people have been asking about exos San they hyper converge V2 will be out soon once it comes out I will do a full review of it but you can go in forums and check out a preview of it today we're just going to focus on NFS ice cuzi of course is an option so if you have a Nas that only supports ice scuzzy it will work but please note ice scuzzy is not thin provisioned NFS is NFS is generally my preferred storage when I'm tying either a single host or in this case Two Hosts in a resource pool to a Nas this particular Nas or TI to is true Nas but it will work with others NFS just works well because it's a thin provision it is file based it is easy to manage so let's go ahead and set that up to set up new storage we go to new and then storage then we select the host we can choose server one or server two this doesn't matter if it's a shared storage we'll call this one true and ass NFS we're going to go ahead and select the NFS as an option the server would matter if we were choosing for example a local one because it would be looking for something local on that one but because this is shared either server we choose in this resource pool is going to connect to both servers provided they have the networking setup so both servers can talk to it and we're going to put in the IP address we're going to query it and that will let us then select the path I have my NFS storage right here default NFS version is fine unless you have custom reasons or need some custom pathing and custom options but I'm not worried about any of that we're just going to leave it all default and go ahead and hit create now it's created that storage repository and we can even take and migrate a VM to it so let's go ahead and take a VM and we don't even have to shut this down this one's currently running on local storage here we can go ahead and migrate this to the trass NFS we'll hit okay and it'll kick off the migration process and a task it will not shut down the VM while it does this it's just takes some time to migrate from the local storage over to the NFS storage and it's going to go as fast as well your server connections are all right now that the storage for our virtual machine has been moved let's go back over to VMS we can click on it here and we can see that it's showing the true nass NF s is where it's living now let's go ahead and look at this particular storage we only have the one disc in it but we do have the Delta that was attached to that disc in here as well so you see the disc you see the Delta attached to it and if we click on it it brings us back to our VM and let's say we go ahead and create some more snapshots and create another one then we go back over the disc you'll see those as well so each one of these is on here now they don't each take 80 gigs because of the thin provisioning hence the reason I said NFS is a good way to go but let's go ahead and look at the Advan and talk about what some of these do we have a reclaim freed space but that doesn't matter under NFS and it won't do anything because that's not needed for this particular resource type the buttons I really want to talk about are remove Sr which kind of as it may sound will delete this dis not something we want to do but that's how you'd remove a storage repository but enable maintenance mode I really like this feature if you have a lot of VMS on this particular server and you'd want to for example update your trass you can take this and hit maintenance mode and we're going to go ahead and hit okay with this puts this in maintenance mode which is going to shut down all the VMS that are running at it there's only one right now so this BM was running and if we try to start it it lets me know no host available because it is in maintenance mode so let's go back over to our storage our churn NFS and once I'm done with maintenance mode we can go back over to advance and we can say disable maintenance mode this may sound trivial but when you have as many as 100 VMS running on here this can be kind of a tedious problem to solve to say how many VMS are on here and I need them to stop so I can update my trass another option as I noted here though is it does not require you to shut down the VM to move these to another storage so if you have multiple nasas and you want to do maintenance you can just migrate them over to another one live and then bring them back after you've done maintenance but I think maintenance mode works really well it's a quick way to shut down all the VMS provide do your updates on whatever your Nas is or whatever reason you had to put it in maintenance mode and once you take it out of maintenance mode it will restart those VMS and you're back up and running now this VM is running on xcp server one what if we wanted to migrate this to server 2 but we don't have to migrate the storage now because when we added this storage because it was a shared type storage it was automatically added to both of these so we can go here and it's the only other option is send it to this server server 2 and we just hit okay now when we do this I want to note that it's going to go substantially faster because it only has to take the memory that is running and move it over it does not have to actually move the storage so now this VM will move quite fast over to the other host now the vm's moved over go back over to Virtual Machine and you can see it's now on XC PNG server to and the storage is exactly where it was still on the chass NFS storage now if you want to create a new VM it's pretty simple to create them on that storage when we go to create new we'll select the pool select any template and then for the storage you can go down here and select where you want this to land and we can choose trass NFS so it works much the same way if you want to create anything new on that particular storage I'll also note if you want this to be the default storage for any new VMS that are created you can go over here to home then storage and we can go here and we choose the set as default so now this is the default storage so anything new automatically will be created here if you wanted to change which the default is we could change it to any one of the other ones in here but we'll leave it right here as the trass NFS as a default storage repository so anytime we go back and create a new VM now it's automatically going to have this one selected right here one more thing to note when we're looking at the storage and we can go here to the trun and you want to see which host is connected to it lets you know the path that these are connected to on each of the hosts if there's ever a problem there is an option to go ahead and connect all hosts we can click that now and it's just going to in case there was a problem or one of the servers was down and you see that these are disconnected it will force reconnect these to all the hosts now let's talk about backups we're going to go back over to home and I want to clean up some things I have on this VM I just don't need all these snapshots let's go ahead and purge them and you can see under backups there are no backups attached to this particular VM and if we go to backups and new we can go ahead and create one but before we go here I want to point out that if we go under settings and remotes remotes are the term for where you want to store your backups I've got this attached to my churness already I've actually got a lot of other backups that are in here and we can verify that it's working by clicking on the little test your remote it'll tell me the speed at which it's able to talk to this particular remote these as I said are not particularly fast machines but it'll get the job done for what we need now we're going we're going to go ahead and go back to the backups and we want to create a new backup there's a couple different options in here first we have the option for EXO config pool metadata backup this will actually back up as it says pool metadata and EXO config so let's go ahead and make one of those and we're going to choose the target remote and I'm not worried if I wanted to I could set it up so it always sends me an email or only a failure or never notifies me but that is in the plug-in settings if you set up the plug-in for setting email and put a mail server in there we need to select the pool that we're going to do this to this backup job and go ahead and set a schedule as well we'll say daily and we can pick some time that we want so let's retain three copies of each of these and whatever time works for you we're just going to ahead and click okay it gives you a preview that this will happen daily at 12:00 a.m. if we wanted to change that we can then say okay we want it to happen at 2 p.m. so go ahead and hit okay you want to enable this if you want it to run automatically I'm not worried about that we're just going ahead and hit create now the importance of a pool backup is if you lose the data in the pool for all the metadata for the network settings you have and all of those other configurations they're all stored right here including the settings for XO now you can actually export all the EXO config right here as well so if we go to settings XO config you have the ability to import a new config or export the old config right here but it's nice that they have a backup option that will let you do that and you can see how quickly this runs we'll run it again just to show you it runs really fast and now we have the ability to have a copy of the entire metadata and the backup now as far as restore goes for this we'll go ahead and hit restore and choose metadata and there is the restore option for this particular metadata we'd select which version we want and we could just push that back and it does break them out this is the XO backup this is the pool data backup now let's go ahead and create a new backup again and VM mirr backup I'll mention is the way you can take two backup repositories and mirror them they got some documentation on it this is a pretty cool feature we're not going to spend much time on that it's kind of self-explanatory that if you have one backup repository you like to mirr to another you can do that this is just mirroring the backup repositories maybe you have one on-site and you like to mirror it offsite or just to a secondary location you would just select the different remotes but because I don't have two selected I'll cover that in a later video talking about how that feature works but as I said it's relatively simple let's go to new and let's do a virtual machine backup and we'll call this one demo Deb 12 backup we have a lot of different options here we've got a normal backup which is just a full backup of the VM or we can do a Delta backup which is a full backup and then Deltas afterwards essentially incremental both of these backup types land on one of these remotes as the ref for two but it's an external storage these ones though such as Disaster Recovery you actually send them to another system so we can actually have it backing up to the local storage or even another Pool if we had more than one pool in here to another Host this is nice if you want to create a backup that is on another machine that you can just click the start button without restoring and run it at that moment when you want to recover that VM same thing goes for doing continuous replication it will replicate this VM onto another host these are kind of unique ones that you may or may not want depending on your needs most of the time people are looking for the full backup or a Delta backup and the Delta backup is going to as I said be a full first and then incremental afterwards so let's go ahead and select that debban 12 VM and you may have noticed here you have the ability to backup more than one system at the same time under one backup job there's also what they refer to as smart modes the tags that we mentioned earlier you can select Things based on on their tag so you can choose certain tags and then let those tags such as Tom Drive the backup so everything you tag is time would then be put into the backup this is good for people who are building VMS and then they want to put them into production you can set a production tag and then any time you set that production tag it'll automatically without going back to the backup back up all of those VMS but we're just going to keep it simple here we're just going to back up this one and we'll set a schedule and I'm going to call the schedule just keep three daily copies backup retention of three and this is the health check option this will actually automatically boot that VM and verify that it works if you'd like it to do that I've got a whole video where I dive deep into exactly how that works it's a wonderful feature to validate that your backups are working we'll set these and we'll just say every day but maybe we want it be weekday so we'll just say every day like this or you can choose the days that you want the backups to run but frequently this may be more popular because people may want the incrementals every day or maybe they don't need them on Sunday days because well no data is being transferred whichever works for you we'll just leave it at every day selected only one of the hours is selected but we could back it up every hour Deltas actually happen really fast kind of depends on your risk tolerance for how much data you want on there they also give you a little slider to make it easier if you want to stagger them around for when the backups are going for ever so many hours but we'll just keep it simple here as I said and do a daily backup and we're going to go ahead and hit okay now I'll turn this off enabled so I don't want it to accidentally run but it defaults to enabled this is also a one any relationship you can have this schedule and then you can add a second schedule to this same backup or even a third and fourth so if we do this click okay do this and click okay you can see that we can have multiple backup schedules with different parameters in them for how we want things backed up so we'll go ahead and stop those we only need this one here for this demo select the remotes we've only selected one but you can actually send it to two remotes at the same time that is supported under advanced settings we have a few more options we have the report settings so we want it to notify always failure or never and we can set specific email recipients it does require that the email plug-in be set up concurrency how many do we want running at once I'm going to leave this blank but you can specify depending on how fast your server is maybe you don't want them all to run at once because you have multiple jobs this will set the concurrency of them especially important when you're backing up a lot of VMS at once we're only backing in one so it doesn't matter if this takes more than an hour I think it should fail and let me know because it must be stuck these generally go really fast full backup interval this is how frequently you'd like it to run a full if not it'll continuously run Deltas but you can set this to something like 20 this links to the documentation that suggest us be a good setting but the every 20 times it reinitializes your Delta to start from the beginning and then you always have that full backup there as opposed to the chain of Deltas it's just an extra Assurance I haven't run into a problem but it is nice that they added this in here use MBD protocol call The Transfer if available we're going to turn this on and I'll show you where that has to be turned on number of Meed connections per disc we'll leave it at one you can rate limit these if you want and finally we have normal with memory or offline you can do these snapshots just like I showed earlier with memory but I like offline cuz I want to stop the VM I don't have to stop the VM but it is nice because this will stop the VM as quick as possible go through the shutdown grab the snapshot and boot the VM back up completely optional to do this we can leave it in either mode but we'll go ahead and leave it right here in offline mode to show you how it works now the NBD let's go over here to our pool and our Network and then we're going to choose NBD connection on E I know e is where this is attached to so this is where the backup's going to occur because this is where Zen Orchestra talks to these servers this is for the network block device transfer this is an enhanced way to allow for faster speeds on Deltas so we go ahead and turn it on let's go over our backup and we can run the backup from here this is our demo de2 backup I don't want it to accidentally run so I turn that back off there's no successful here because it's never done it lets me know it's three daily copies I can click this to see the matching VMS I can just run the backup job once here I can click to see the matching VMS we can also while we're looking at the VMS here because it's attached to this or any VM it's attached to it also now shows up under the backup tab let's go ahead and just kick this off and run it and click okay it's shutting down the VM which happens rather fast now it's booting back up now the snapshot is exporting over to the remote so it's doing the backup now now the backup is complete so we can go back over to our overview and we can see that it transferred 3.72 gigs which is the size of that VM took about 2 minutes to do and if we click on it it gives us the details of the transfer data started using MBD it did the snapshot then did the transfer the duration was 2 minutes to get the transfer tells you the speed and then the type and full so we're all set and there's only one VM here so there's no other information available for this particular VM and we know the backup is successful now we can go back over this VM and look at it and we see a snapshot created when you do the Deltas it leaves a snapshot there so this snapshot will be updated every time this Delta backup is this is actually a nice snapshot point that we can revert the VM to the VM is currently up and running let's log into it let's do a quick appkit update and an appkit upgrade there's nothing changed but we made a few changes on it so let's go ahead and run the backup again cuz I want to show you the differential now because there's not much change so this will go even faster the second time so we're going ahead and run that same back up we go over here to task it's going through the shutdown process and now it's back up and running and transferring that data back over and complet it I did this all in real time without speeding it up up so if we go back over to our backup and overview there was only a 176 megabytes of change so the whole process took 16 seconds so from snapshot shut down transfer backup and system being started back up and the transfer that little bit of data only 16 seconds passed so the backups being natively integrated with Zen archestra are really nice because they work so well and so fast because this system knows exactly only what block have changed since we last booted that system of course the more data in the system the longer it's going to take to run the Delta because it's the difference between the last backup but you can see the efficiency of it running this way now as far as restores go to here to restore there is our Debian 12 system if we wanted to restore it we could just go here select the version we want select where we want it to land I can leave the current VM there and we can even say let's send it to the local storage of server one do we want to start the VM do we want to generate a new Mac address do we want to use a DI differential restore differential restore is interesting because we can actually restore it differential if it was in the same repository where it can take the blocks that are already there and just figure out what the differences are without having to do a full restore if you're restoring larger VMS into the same place this can be a great way to do it if you wanted one of the older versions but let's just walk through the restore process and we're going to dra a new generate a new Mac address that way it'll get a new IP address because it'll technically be like a new VM although they will have the same name and we're going to send it to local storage but we could just as easily send it to any one of these so hit okay and we can look at the task while it completes now the virtual machine's been restored so we can go over here to VMS and you'll see that it has the date that we did The ReStore in it and there's going to be a tag that says restored from backup so we know that this one restored but if we go ahead and start it it'll work perfectly fine and because I did tell to change the MAC address it won't have a conflict where it tries to get the same IP cuz it has a new Mac by the way if you ever want to change a MAC address just click and you can type a new one in now we can look at the console this one's booting up and if we want to look at the restore logs we can go to backup and then restore and it has the details right here that this was restored took about 3 minutes and 26 seconds to complete the restore and if we wanted to see a raw log they have an option right here to give you more details now some of you may have noticed this little Health button over here this lets you know when there are detached backups there are backups that don't have jobs listed to them it's pretty simple Health Report and it'll say well for some reason there's a detached VM snapshot that's located here that doesn't seem to have a job related to it you can remove the snapshot it will delete the snapshot from that particular VM so there's a snapshot here but there's no backup so we can go back over to the backup and health and if we want to delete it we can because there's not a job related to it but do note it is deleting that snapshot so we'll go ahead and get rid of that and now the health is back to no warnings now while this video is enough to take you from loading Zen to getting a VM setup I do have several videos as I mentioned that are all linked down below also in my Forum post of deeper dives into specific topics also take the time to read through the documentation they have quite a bit of it they have a lot of visuals and Graphics in there to help explain things and the documentation is constantly being updated as the project progresses there's always new features they do a monthly run of all the new features and you can check out their blog post to see this over time they've been doing this for years it's actually really cool to see the development and stay up with the once a month announcements for all the new features that they add each month both to XC PNG and Zen Orchestra and of course I'll be doing an updated video when the Zen Orchestra 6 comes out along with exolite because that will be a facelift as well as I mentioned the EXO store when that comes into full production the version two that will be their hyper converge platform and I'll be talking about that as well and I'll keep those videos up to date in this particular video so you're always able to find them and I have a playlist of all of my xcp and GN and orchestra videos that's also linked down there so if you have questions comments or concerns leave them in the comments down below or head over my forums for more in-depth discussion like And subscribe to see more content from my channel head over to the forums that are by xcp as well the Vates team is very active in the forums and I highly recommend participating with them and participating in development and engaging with them this is one of the things I really like about the whole team is they're very engaged with the community and building it out to work for people's different scenarios all right and [Music] thanks

Channel: Lawrence Systems

Views: 47,418

Keywords: LawrenceSystems, xenserver tutorial, xen orchestra, open source, xen server, xen orchestra community edition, xcp-ng xen orchestra, xen orchestra tutorial, xcp-ng backup, xen orchestra backup, xcp-ng tutorial, xcp-ng install, Getting Started XCP NG 2024, xenserver 8 installation, xenserver 8, xenserver installation, xenserver vs xcp-ng, xenserver install vm, xen orchestra create vm, xen orchestra quick deploy, xen orchestra lite

Id: 2wMmSm_ZeZ4

Length: 66min 52sec (4012 seconds)

Published: Sat Feb 24 2024