How to add a cache drive, replace a cache drive or create a cache pool

hello guys and welcome to another tutorial so you have your server set up and everything works but you didn't bother with a cache drive at the time so now you're thinking it's time to benefit from the quicker write speeds it offers or maybe you have a cache drive but now you want to upgrade it to a larger one or maybe you want to get really advanced and set up a cash pool so let's jump in and see the best and easiest way to get this done so before we start let's firstly talk about why I have a cache drive at all well with a cache drive we can massively speed up the array what we can do is we can set up shares that write their data to this fast cache drive first and then later on when the server isn't busy the server then moves this data from the cache drive over to the parity protected array so anyway why is writing directly to the array so much slower than writing to a normal disk well most people will tell you it's because it's having to write to a parity disk as well but whilst this is correct it's not the whole story the fact is is to computer actually has to read the data before it can write it and it uses a technique called rmw which stands to read modify write and what this is is when the write is sent to the array the array first reads the sector's on the data drive where the new data is going to be written and it also reads the parity locks on the parity drive that relate to that data so that's the read part so now with those two reads it can calculate the new parity for the data it's about 2 right so that's the modified bit but before it can do the last part the write bit it has to wait for the platter to rotate back around to bring the block back into the correct position so it can finally write the data and the new parity blocks so it being slower is due to the fact of two different drives being written to it's basically everything that has to happen beforehand so a cache drive is good for fast writes to the array but it also can be used to store data the you want the fastest read and write speed for such as to store VM images or Dockers and this gives some far better performance now the best drives to use for cache drives have to be SSDs not only just for their speed but also they have no moving parts and they consume far less energy remember chemical counterparts so this makes some ideal candidates for a drive that's going to have a lot of heavy use but with a cache drive we must think about should the drive fail do we have a backup of the data especially if we only have the one cache drive in our server and yes we can have more than one cache and actually set them up in a raid configuration to easy redundancy or striped for even more cache speed so in this tutorial we're going to do the following one add a cache drive to an existing server that doesn't have one to replace a cache drive with a larger one and three add a second cache drive creating a cache pool in a btrfs raid configuration ok so on my test server I only have two drives assigned I have a parity drive and I have one data disk as you can see there's no cache drives but I do have two unassigned devices here I had a Shiba hard drive and 60 gigs and a Hitachi hard drive at 250 gigs so let's just have a look our shares these are the shares that are on the server now when we put a cache drive in we're going to want some of these shares to be on the cache so let's have a look at what share should go where the shares that should be on the cache drive all the time or the app data which is used for data for dockers the domains which is used for our VMs and the system share which is used for things like our docker image and some system files for the VMS to run so some shares do best on both the cache drive and the array and these are called cache enabled shares and so what happens is first the data goes on to the cache Drive and then later on it's transferred over onto the parity protected array so any share where you often write data to it's best to have it as a cache enabled share and the download share is one example of a shared that's like this and other shares that are seldomly written to keep them on the array only there's no point having things like media shares that you normally read files from on a cache enabled share okay so let's have a look at the shares that should be on the cache drive let's look at the app data as you can see here we've got an option that says to use cache disk so we have no that will keep it off the cache all together we have yes that will keep it on the cache drive but it will also be on the array and we have only well we only have it on the cached drive and it will never be moved to the array and we have one here called prefer now we'd prefer it will store it on the cache but then if the cash runs out of space it will then continue to put it on the array and also we have this preferred we don't have a cache drive it will store it on the array and then if a cache drive becomes available when we run move it will then move it from the array onto the cache drive so this is what we want to set all our shares to which we want to have on the cache drive when we add the cache drive in a few moments and we need to check that with the same thread the main and the same for our system so let's add our cache drive so to add the cache drive we're going to have to stop the array so with the array stopped you choose one of your other side drives on the server I'm going to use the Toshiba drive here so now all we need to do is just to scroll down and click on to start array and as we scroll down here you'll see the cache drivers are mountable this is because it's not been formatted so all we need to do is tick this box here and click on the format and as you can see that's been done and by default if it's formatted in btrfs if we look inside the cache try there are no shares here and that's because they still on the array and what we need to do is we need to move them here on to the cache and to do that we need to scroll down and then to click on the button move now and this will move all of our shares that we have set as prefer cash on to the cache drive ok now you can see the move is finished and if we click on it again we'll see it will just run and there's nothing for it to move so let's have a look at Li discs we can see here that the app date is no longer here and the domains is no longer here but why have we got this system folder still here and we can see in here we've got our docker image here and we've still got the libvirt image here as well so let's have a look at our cache Drive and we have a system folder and inside we've got a docker folder and we have our live that folder well the reason it hasn't copied across is because these files are in use that presence because with our Dockers and of the EM is enabled so what we need to do is we need to go to docker and we need to click on here and set the docker and able to be no and we need to do the same for the VM manager we need to set that to so now those files won't be in use so now we can go down and we can click on move now and we'll see the movers running again so now that we'll move those files across and then when the files have moved successfully across we can we enable docker and re-enable the ends ok move is now finished so let's have a look at our cache drive now and now if we go back to system will see that our docker image has been moved across and our lives our image has been moved across but now what we do as we have our cache drive set up is we go to the download share here and we would enable it to use the cache drive so now it's able to use the cache drive as well so we could click on apply and click on done ok and so after having put the cache drive in we can now go back and we can start up a docker and after this run one of the darker sister check it works let's just check the Doppler works can ambush work inside okay so the maybe time so we want to replace our cache drive with a different drive and that's really easy to do but what we have to make sure we do is that we copy all of the stuff off our cache drive before we remove it from the server so to do that the first thing we need to do is make sure we have squid community applications plug-in installed and if you don't know how to do that then go to the line technology forums and scroll down till you come to application support and then go to plug-in support here you'll see the plug-in community applications scroll down to here where you see this line of code and copy this location and you can close this and click on to plugins and click install plug-in paste that into there and click install and that will install the plug-in and now you'll see that we have an Apps tab there's going to be a dock are also going to need so we'll install that now and that docker is a file manager called crusader so let's do a search for crusader and click on to add that should be on /mnt forward slash and then for the host part 2 we have the NR updater folder and /ke e and then just click apply it so then pull down the stocker and install it okay so when that's done you have all the software that you need what we need to do is to set up a couple of shares we're going to backup the things that we're going to use so first let's let's make a share and we'll just call it can update the backup and that will back up our app data and obviously you don't want to use the cache drive and our second share we'll just call it the M back up and again of course we don't want to use the cache okay so we've got our two backup folders created so now the first thing we're going to do is going to click on settings and now under the community applications you see we've got a backup and restore tab first thing we need to do is fill out where our updates are is so click on to here and click app data if you need to save this to a bisque or to a share so we're going to click to a share so we're going to have this onto user and the destination share click here and we'll could choose the one we've just created here app data backup and if you scroll down here we can also backup things such as our flash drive but we don't want to do that right now and also backing up our VM xml's we're not going to bother doing that either and so that's all the things we need to do on here so now click on to apply click OK and now click backup now okay now that's complete we look at our shares which is see in here now that we've got our two app data folders in here so you can see we've copied this folder here so now we want to go on and do our DMS in your domain and so for that we're going to use the medaka we just install some of them use Crusader so click on here then go to web UI and then you'll see two columns of files and if you go down here you'll see on raid and that's what we mapped to earlier to our /mnt forward slash so if we click on here we can see all of our disks and our user shares so we want the copy from the cache drive so let's click on to here and now let's go on this side and find our user share and our the end backup and now what we're going to do is it's going to drag this across here our domains and drop it here and just click copy here so with that copied now we're going to go across the settings and we want to go onto our VM manager and then we want to disable the VM manager by clicking on two nodes and now we're going to go back to crusader and we're going to go back into the system file here we're going to coffee across the lid vert folder and copy it to here ok so now that's copied and in the system file is also the docker image here but we're not going to copy this because it can cause problems coughing this file sometimes so what we'll do is we'll recreate that later so now let's take a look inside the cache drive so we've copied the app data we've copied the domains and we've also copied the libvirt in the system folder and so with everything copied we can now replace the cache drive and so it's now time to go down and to stop the array and at this point you'd then shut down the server remove the existing cache drive and put your new cache drive in as I've already got another drive in my server I'm not going to do that base I'm just going to go to the bit where we swap them over the cache so just under the cache section just selecting you drive and then start the array and as it some you drive it says insurmountable so all we have to do is just format it so it's click here and click format and now I'll cast drive is 250 gigs but if we have a look here obviously we've got no folders here so the first thing we want to do is we want to go to our settings and we want to go to our backup and restore so now we want to click on the restore app data tab and then let's go down and click restore and ok and then that's complete so now if we go back to our main tab and we look on our cache we see now we've got the app data in here fine so now we need to rebuild our doc an image in the new cache drive so go to docker and then click on to know to disable the Dockers and apply and then done and now click delete docker image file and then click on to apply and done and now we just need to start the docker engine again so go back to docker and click on that and then click on to yes and then apply and then done and so now if we go back and take a look inside the cache drive and see the files there we'll see that we can see the system file now with the docker image inside ok so now it's time to put the Dockers back inside the docker image so for that click onto docker and you'll see there's no Dockers here at all so I want to do is click Add container and then click on select a template and then in the list you'll see this one since in my and this once has been xmv the one I'm going to choose so any template that supplies the one you want and then scroll down you'll see it's already filled out for you and then click on to apply it will pull down the docker and everything we're back how it was so now that's done and you can see that the MB Dockers back here make sure you do that for all of your Dockers I'm just going to quickly install my other block of this Crusader locker that's going to need that now so just click on so your other Dockers and install them ok so once you've restored your Dockers and go to your crusader docker and startup that and so now we want to go back to our unread share here and go back to our user share I'm going to go towards the end back up here and you'll ok right here and then go to our car drive here with our system data just pull this across to here our domains needs to go into our cache drive into this area here so we just wanna drop this into here and click copy here ok so now that's moved across and so now with the domains moved across we can go to our VM manager and reenable VMs again and then test at the end and now if we go to my VM tab you can see we've got the fedora IM still there because all the xml files are copied over inside to live that image so let's just check this actually works as it should work fine because in the libvirt file it also has the env Rams for the OU VMs and this being a no VMs VM it should be fine because we've copied all that over now if we haven't have done that and only copied xml and this machine wouldn't boot ok so now let's add a cache drive to the cache pool so we've got our existing cache drive here a 250 gig drive and there's another term transistor gate right here this unassigned so I'm going to use this one and add it as a cache drive creating a cache ball so to do that we need to first stop the array and then where it says slots we want to add a second slot through our second cache drive and now we're going to choose the other Hitachi drive and now we just need to stop the array again so with the array started again we can see the two hard drives here with a total size of 250 gigs and even though they've got to turn from the 50 gig drives it's automatically set up in a RAID one giving us a mirrored raid so should one of the hard drives fail we'd still be ok but you don't have to keep it as a RAID one you could make it a raid 0 and strike the data across both the drives and this would give us much better performance but then would lose the redundancy so to change the raid level just click on the top cache Drive and it will bring us into the settings for that you'll see now we can see the pool information and here you can see that we have a raid 1 so it's a mirrored raid and the balance status at the moment balance is running and we have 81% mass until it's finished so if I was to let this finish I'd have a fully mirrored cache but I don't want to do that so I'm going to click cancel I want to change the raid level and now you'll see here next to the button that says balance we can actually put in a command that will convert the raid into the type of raid that we want and these are the various different ways that we can have online raid so you can see in the list above all the different raid levels that we can use so all we have to do is basically type that command into the box next to balance and I'm going to use the one that's like j-pod and type that one in there so I'm going to put single here but you can use whatever raised level you require and once the commands in there just click on the button that says balance and it will take quite a long time for it to balance to drive and to be complete but then after which you'll have the raid level that you require ok so now the balancing is finished and you can now see that the two cache drives together now come to a total size of 500 gigs we've got 461 gigs free and they've used 38 gigs so guys that's all about the different ways we can use cache drives in unread and that brings us to the end of another video well guys I really hope you enjoyed it and you found it useful so if you did then please hit that like button and if you're not already a subscriber to the channel then please please hit the subscribe button and subscribe and anyway whatever you're up to for the rest of the day I hope it's good and I'll catch you all in the next video

Channel: Spaceinvader One

Views: 118,097

Keywords: unRAID, unraid, linux, nas, home server, cache drive, dockers, virtual machines, vms, lime technology, lime tech, limetech, how to replace cache drive, how to upgrade cache drive, cache pool, btrfs raid, unraid array, speed up array, How to add a cache drive, replace a cache drive or create a cache pool, tutorial, guide, how to, unraid tutorial, unraid disks, unraid cache, parity, unraid parity

Id: ij8AOEF1pTU

Length: 19min 47sec (1187 seconds)

Published: Sat Jan 28 2017

Reddit Comments

[u/spaceinvaderone]

Hi guys this week i have made a tutorial about how to add a cache drive to your server. It also shows how to upgrade or replace an existing cache drive without loosing data Also you will see how to create a raided btrfs cache pool. Hope you find it useful! :)

[u/hadesteuh23]

Great vid as always! Keep them coming really useful!!

[u/MrChunky]

Great video as usual! I was wondering, what's the advantage of using your method, rather then disallowing all shares from using the cache drive, then using the mover. Shutting everything down and replacing the drive. Then reenabling chache for the relevant shares?

It seems that this method is proposed in unRAID documentation if I am not mistaken.

[u/kking112391]

Great content as always. These videos have also helped me out in a pinch before. Keep them coming and I'll keep watching them!

[u/one321]

I plan on upgrading my pooled cache drives this weekend. Timely video. Thanks!