All about Using Multiple Cache Pools and Shares in Unraid 6.9

hi guys and welcome to another video in the unraid 6.9 series in this video we're going to carry on setting up our unraid server and we're going to be looking at a new feature where we can set up multiple pools so not only are we going to be looking at how to set them up but also why and what we can do with them so we're going to be looking at multiple pools how to use them correctly with our shares and how to make sure our shares are set up correctly so if that sounds interesting then let's get started okay so this is a new install of unraid running on raid 6.9.2 and the hardware in this server is it's an amd server with an asrock x470 taichi ultimate motherboard a ryzen 9 5950x cpu and 64 gigs of ddr4 ram and the array here is made up of eight eight terabyte disks one of which is parity so giving a total here of 24 terabytes in the array now it has one pool device here the cache drive which is a regular ssd which is one terabyte in size as well there are four unassigned disks in this server as well two of which are nvme drives which are two terabytes each and another two regular hard drives which are one terabyte each now if we click onto shares here these are all the default shares which onraid sets up there's app data which holds our data for our containers there's domains which holds our data for our vms and the iso share is the share where we put our in-store media to install those vms and the system share well just like the name says stores our system files and if we look at the users tab there's just one user at the moment the root user will be setting up other users here which we'll use for the security with the shares obviously there's no containers installed on the docker tab and also there's no vms installed on this server either so before we actually go ahead and install any docker containers or vms we want to plan exactly what we're going to do on the server and so how we're going to set up the shares and just what we're going to do with the unassigned disks at the bottom here well until unraid 6.9 was released all we could do is use them as unassigned disks but now since unraid 6.9 we can use these additional disks and make multiple different cache pools and yes having multiple cache pools that will increase the performance of the server but in fact you don't actually have to have a cache drive you can run an unraid server just on an array but it's highly recommended to have at least one cache ball to improve the performance of both your docker containers vms and the write speeds to your server but before we go and set up some new pools i think it's important that we have another look at the shares tab here and see how all these shares work so we can plan the best way to use our multiple pools so what we want to look at for the default shares here is this part here where it says cache we can see here it says prefer cash for these three shares here and this share here says yes cash so basically where it says prefer cash here when you set a share for the setting prefer what unraid will do is try and keep all of the data in that particular share preferring to keep on the pool device called cache now the only time it won't do that is if this pool becomes full should the pool become full it will then put new files onto the array but then as space becomes available on this pool later on when mover runs it will move the stuff that it put temporarily onto the array back onto the cache drive so that's why on this share we see prefer and cache so that's why we see app data preferring to use a pool device because this stores all of the persistent data for our docker containers so it's best to have that on an ssd which normally our pool devices are and the same here for domains this stores our vmv disks so you want them to be very fast as well preferably running off an ssd so this is set to prefer cache as well and also this system share this keeps our docker image inside where all of the actual containers for the docker containers are written so you may be wondering what's the difference between the data that's written here in the docker image and what's written in the app data well the data in the docker image can change like when you update the container to a newer version the docker image is updated but the app data isn't updated when you update the docker container because the app data contains all of the persistent data which needs to stay the same things like the configuration for that particular container think of it a bit like when you have a video game on a pc or a console the actual game itself is kind of like the docker image you can update the game if there's a new patch that comes out to update it because those parts are the same for everyone and so the app data's like things like the game saves how far you've got in the game maybe you've customized a character so it's all of the data that has to stay the same but it's personal only to you so the docker image contains all of the universal container images and the app data all of the settings for that container that are personal to you and there's something else in the system file as well there's something called the libvert image which is used to store various things in to do the vms okay so we can see why shares that are set to prefer want to have very fast io but what's this other setting here where it says yes and then the name of the pool here cache what does yes mean and how is that different to prefer so when a share is set to yes what it means is that it will use a cache pool for all new files and directories that are written so that will make the rights very fast and the reason it says yes cache is because that's the name of the cache pool that it's going to use to do this so how this setting is different from prefer is because this time when mover runs mover will move all of the files that have been written to that particular share over onto the parity protected array so yes it's similar to the other setting where it writes the new files to the pool but it's different what it does with them when mover runs if the share is set to yes when mover runs it will move it to the array if the share is set to prefer mover does the opposite it moves files from the array onto the cache and we've also got the setting here only where it will only use the cache for new files even if the cache becomes full it will still keep on trying to write it there so really i'd never use that myself and i think this will probably be a move for my raid and future builds as this prefer does the same but it's much better now the other setting obviously is this one no where it doesn't use any of the pools for new files it will just write directly to the array okay so these share settings obviously have a lot in to do with the pools so what we're going to do is we're going to set up some more pools for some of these shares so obviously the app data and the docker image in the system file well that's in to do a docker and the domains and the libvert file in the system file is in to do with vms so i'm going to set up two separate pools one that i use for vms and one that i use for docker so to do that we need to go to the main tab here and actually stop the array so with the array stop we're able to make changes to the pools now before i go and add a new one i'm going to rename this one here so to do that i'm just going to click onto its name here but i just want to draw your attention to one thing first if you look at the name of this pool it's cash and it starts with a capital letter but if you look here everything's lower case it only gets capitalized by the web gui display on the main tab anyway so if we click on the pool's name here we can change it if we want to and i'm going to change this pool's name to reflect the type of drive that this pool is on so i'm going to rename this pool to cash underscore ssd and then click onto the rename button to confirm the change and then click done okay so back on the main tab if we scroll back down we can see the name of the cache pool its name has changed correctly but being displayed in the gui is being displayed with a capital c so let's add another cache pool now so i'm going to click on to add pool here and this pool i'm going to call cash underscore nvme and next we choose how many drives we want to make up this cache pool and we do that by assigning the number of slots i only want this pool to be one drive so i'll just assign one slot so once the pool's been named and we've defined the number of slots we just need to click the add button here okay so now let's scroll down to see that pool we can see it here cache nvme so now we can choose the disk that's going to be part of this pool at the moment it says unassigned but clicking onto that i can choose a drive from the available drives left on the server so obviously if i name this pool cache nvme i'm going to choose an nvme drive so i'm going to choose between these two here i think i'm going to choose the western digital now because this pool only consists of one drive i can actually choose the file system that this pool will use by default obviously it's set to auto and so the default file system for pool devices is butter fs so if i were to leave this on auto that's the file system that this pool would be formatted in and the reason that butter fs is the default file system for pulled devices on unraid because it's the only file system here that allows us to use multiple drives in different btrfs profiles such as raid 1 raid 0 raid 5 etcetera but because i'm only using one drive in this pool i'm going to choose xfs for this pool's file system okay to confirm that change i'm going to click on to apply and then done now i'm going to add another cache pool so i'm going to click add pool again again this pool i only want it to have one drive so i'm just going to choose one slot and i'm going to name this pool vms underscore nvme and for this pool again i want it to have an nvme drive so i'll choose this sabrent one here and again i think i'm going to choose xfs for this drive as well okay so lastly i'm going to add one extra pool in which i'm going to add two drives to this particular pool and because i'm adding two drives the default file system will be better fs this time and the pool by default will be created in raid one so that will give me redundancy should a drive fail so because this pool's got redundancy let's call it cash underscore protected and i'm going to set the slots to two this time and then assign the first disk to slot one and the second disk to slot two okay so that's both of those disks added to this pool and these two disks in fact aren't actually ssds these are just normal mechanical discs you can assign mechanical discs ssds or nvmes to a pool but unfortunately when making this tutorial i didn't have any extra ssds to actually demonstrate so i'm using normal mechanical okay so now if we click on this two disk pool here like i mentioned earlier when there's multiple disks the only file system we can use is btrfs or btrfs encrypted so i'm going to leave it on auto which is btrfs but the non-encrypted version okay so those are all the pools assigned that i want to set up so now i can scroll down to the bottom and start up the array and when the array starts we're going to get prompted to format the disks it will say that they're unmountable so all we need to do is scroll down and check this tick box here to allow formatting and of course with the format anything on the disks will be erased so with the warning out the way let's click on to format and then wait for our unraid to format all of the drives in the new pools okay so now all of the pool devices they've been formatted and now this server has four different pools so obviously the fastest pools are the ones that are on nvme drives this cache nvme here now this i'm going to use to put my docker images on and my app data on the other nvme pool here is the vms one so obviously this storing all of the vdisks for the various vms i'm going to set up and this pool here the cache ssd obviously this is the original cache drive that i had in the server this pool won't have any docker images any app data or any vms on it i really want to keep the i o separate for all of those different things so what this pool will be used for is for accelerating rights to various shares in the array and i'm going to have a separate download share which uses this drive so all of the downloads will go straight to that drive that ssd which will have a number of different advantages and the most obvious one being is it's not going to be writing directly to the array so it's going to save wear and tear on my main drives as well if you're someone that does a lot of downloading you don't really want lots of i o rights going to your main array and you certainly don't want it going to where your docker containers your app data or your vms are running so keeping it separate will really help the performance of the server add another reason for keeping where we download stuff separate is because people who download a whole bunch of stuff all the time sometimes it can actually fill up the cash drive before mover has a chance to move it when it runs so you might be thinking well what's so bad about that well nothing unless it's the same location where you also store your v disks for your vms your app data for your docker containers or the docker image because the docker and the vm service don't like it if they run out of space where they keep their files it can cause them to crash so keeping things separate stops us having to worry about that happening so now let's look at this last cache pull i've got here the one with the two disks where i'm protected should one of those disks fail well this pool is also going to be used to accelerate rights to shares but only very specific shares so any share where it would be really bad if you were to lose that data before it was written to the parity protected array when mover ran and a prime example of that is a next cloud share and for you new users who don't actually know what next cloud is well it's a self-hosted cloud service kind of a bit like google drive dropbox that kind of thing so when we're running nexcloud it's really important that we get good fast rights but at the same time we don't want to risk losing any data should anything bad happen before mover runs so a raid 1 butter fs pool is perfect for that now obviously it'd be much better if these were ssds so for the sake of this video just imagine they are now i'm sure many of you are thinking well isn't it better to have all of the pool devices set up this way with two disks or more that way the vm pool would be protected and the pool where i store the app data that would be protected too and maybe in an ideal world that would be the best way to set it up but sadly we don't live in a perfect world and we have to make the best of what we got my motherboard only has two slots for nvme drives and i've got two two terabyte nvmes so i can choose to have two two terabyte nvme pools or one two terabyte nvme pool that's protected so for me personally i'd rather have the two separate pools than have a total of four terabytes usable space and anyway we can have the app data automatically back up using an excellent plugin called ca backup restore app data and similarly we can actually back up the vmv disks as well with another plugin so for me personally i don't mind if one of the drives fails and i lose some data because i can always restore to a recent backup right so let's move on so we've got the four cache pulls all set up so now we need to make some changes in the server to be able to use them how we want so let's go back over to the shares tab here because the cache pools these shares are using they're not correct and as well even the cache pool that's mentioned here i've even renamed that to cache ssd anyway so all of it needs changing okay so the app data let's click onto this here i'm going to leave this on prefer and the cache pool i want it to go to cache nvme all of my app data i want on there so i'm going to click on to apply and done okay so now that's on cache nvme now for domains where the vdisks are stored i want to change this to so i'm going to click onto this here and choose vm's nvme here and click on to apply and done okay so that's two of the shares done so for isos i'm going to make this be cache accelerated but for this one i just want to use the regular ssd cache just to accelerate rights to this share and i'm going to click on to apply and done and now finally the system share which has the docker image and the libvert image in i'm going to put this on the nvme cache where i keep my app data okay so now i'm going to click on to apply and then done and now there's one thing we need to do just to get everything perfectly correct if i go into the main tab here and i look on the old cache here you can see here i've got an app data folder a domains folder and a system folder here and these folders were all set up by unraid when the server was first created so now i've made new cache pulls and changed where i want data to go these aren't relevant anymore but luckily because the has only just been set up there's nothing in them so i can just go ahead and delete them all so to be able to do that i just need to make sure the docker service and the vm service are not running so i want to go across the settings here and go to docker i'm going to turn the docker service off and also i'm going to do the same under the vm manager i'm going to turn the vm service off as well now the reason i'm doing this is because i want to delete the folders here and if i hadn't stopped those services then files in the system share would have been in use so what i'm going to do is i'm going to copy this location i can see here which is the location of my original cache drive and i'm going to open a terminal window and i'm going to type rm space hyphen r now this command will delete everything all of the folders recursively in the location that i specify and so now i'm going to paste the location of where those files are and then put forward slash and then an asterisk so that will delete all of the folders here okay so that's done so if i refresh the page now we can see there's nothing here okay so now it's time to go back to settings again and i'm going to go to docker settings here and you can see here now it says that one or more paths don't exist now don't worry about this we can leave it as it is now because we're setting up the docker service again here we can see a new feature of unraid 6.9 we've now got the option to set up where our docker containers are stored we can use a btrfs vdisk an xfsv disk or a directory so you may be wondering what one to actually choose and what you choose here has no bearing on how you formatted the cache pool if your cache pool is btrfs doesn't mean you need to choose a btrfsv disk for your docker image and the same goes to xfs you don't need to choose xfs just because your cache drive is formatted as xfs so the main choice here is whether to use a v disk or use a directory to store your docker containers and i guess there's advantages and disadvantages to each using a v disc you choose the size for that v disk and the more containers you install the more this gets filled up so having a limit on its size can be considered an advantage because if you happen to misconfigure a container and accidentally have it writing data within the actual docker image itself then you'll probably find this out pretty quickly when raid warns that the docker image is becoming full and so because of that warning you can then find out why that's happening and take the appropriate action so while this is an advantage you could also say it's a disadvantage as well because if the docker image becomes full then the docker service will actually stop working but yes you can always increase the size of the docker image that isn't a problem but the advantages of using a directory is that it's not constrained by a certain size for example if you only have one docker container on your system then your docker directory will only take up a small amount of space because that one container is there only however if you installed that same container on a v disk as your docker image because it's installed inside an image that image when it was first created will take up on the cast drive the full space of what its size is so a 20 gig docker image file will take up 20 gigs on your cash drive no matter if it's full or empty of containers okay so what should you choose well basically if you're a beginner i would say choose a btrf sv disk we've used them on onraid for years and they work well but if you're an experienced unraid user then why not use a directory okay so when you've chosen what you want to use then start up the docker service then we want to go to the vm manager and start up the vm service as well again ignore where it says the paths don't exist that's just because we deleted them a moment ago so set enable vms to yes click apply and then done so now let's go back to the main tab here let's have a look at this pool here the cache nvme okay and here we can see we've now got the app data folder and the system folder that's good that's correct okay so now let's see what we can see in the vm's nvme pool okay good so as expected we've got the domain share so now if we look at the original cache drive the cache ssd there shouldn't be anything there at all okay so that's good so obviously everything's set up correctly now all of the default shares are working correctly and they're using the correct cache pools okay so now it's time to move on so now we need to start thinking about what shares we should create for what we're going to be using on the server but that's going to be in the next part of the android 6.9 series in that part we're not only going to set up various different shares we're going to think about when we should have a share cache enabled and the times we shouldn't have a share cache enabled and why we're going to see examples of different shares using different types of cache pulls we're going to be thinking about whether we should let shares use all of the disks in our array or maybe only just some of them so basically we're going to be planning the shares on our server to be the best they can be but like i said that's the next video so if you enjoyed this video then please give it a thumbs up subscribe to the channel if you haven't already done so and share the video with anyone who you think might find it useful to all of my patrons and supporters who might be watching this video thank you so much guys for your support i really really do appreciate it anyways it's the end of this video it's time for me to go but whatever you're up to for the rest of the day i hope it's good and i'll catch you in the next video

Channel: Spaceinvader One

Views: 34,434

Keywords: unraid, home server

Id: jgRSr7yBZfs

Length: 22min 50sec (1370 seconds)

Published: Thu Apr 22 2021