How to Safely Shrink an Unraid Array

hi guys and welcome to another video so we all know how to add this to our unread server but this video is going to show how to remove a disk and shrink the array and we'll be looking at two different ways to do this so if that sounds interesting then let's get started [Applause] so recently i found myself removing a couple of discs from my unraid array so i thought that a good opportunity to make a video about the different ways you can actually shrink an unraid array so of course both ways involve removing one disk or more but there's two different methods which we can employ to get this done the first way the quick and easy way which is we take the disk we want to remove make sure all of the data on that disk is copied off onto some other disks in the array then remove the disk make a new configuration and then rebuild the parity now the downsides of this method is that after we've removed the disk from the array then obviously the parity isn't correct so we have to rebuild the parity during this time the parity is being rebuilt should we be unlucky enough for a disc to fail or we've got no way of rebuilding that disc so although the risks of this happening are probably quite small it still is a possibility now the second way we can actually remove a disc from the unraid array without breaking the parity so this way is probably better if you just really don't want to risk at all having any time where the disks in your array aren't protected by the parity maybe your data is just super important you don't have any other backup so you just can't risk that chance that a disk may fail during a parity sync so in this method again we move all the data off the disk we want to remove onto other disks in the array but this time before removing the disk we zero out the whole of the disk writing zeros to every sector on the disk and by doing this it means when we remove the disk from the array it won't actually affect the parity now if you want to know why it doesn't affect the parity and i suggest you watch my last video before this one where i go in depth about how parity works on unraid anyway so after the disk has been zeroed we remove the disk make a new configuration and this time when we start the array we tell unread that the parity is already valid and with the array started back up all of our disks are protected from failure because we didn't need to rebuild the parity okay so those are the two different methods so now let's go and look at that in practice and obviously we'll start at the beginning looking at the first method where we have to rebuild the parity okay so here i am on my main unroad server and i've got a 72 terabyte array here with 35 terabytes used and 37 terabytes free now i've decided i don't want to keep this first 12 terabyte drive in the array anymore i want to use this drive somewhere else and removing this 12 terabyte drive well i'm still going to have 25 terabytes of free space and that will take me a long time to fill up and also by having fewer drives in the array well that's one less drive that can go wrong and also my server is going to use a little bit less electricity so it's a 360 win all round okay so at the moment if i look on this disc it's got about 400 gigs of data on it if i go to the right here and click on view i've got some isos here in the iso share and also some software in the software share so i need to move these across to another disk and then that will allow me to remove this disk as there'll be nothing on it and to do that i'm going to use a plugin called unbalance so i'm going to head across to the apps tab and do a search for unbalance so i'm going to install this plugin okay so now it's installed i'm going to go to the plugins tab here and scroll down and i'm going to click on the icon here to look at the settings and here where it says enable unbalanced server i'm going to change it from no on to yes and now you can see at the top here it says open web ui now the web ui address is the address of your own raid server and then the default port here six two three seven if we click onto this link it will open it straight away and so here are the disks in my server and it's this disk here i want to remove now you can see at the top here the unbalance is set onto scatter so what this means is unbalance will scatter or move data from a source disk and we choose the source disk by ticking here on from and it will move or scatter the data from the shares that we tick here so obviously i'm selecting all of the shares on that drive and it will move the data to the destination disks which are selected under the two column the drives that have a tick and at the moment it's selected just to spread the data over the other existing disks i don't want it to do that i only wanted to put it on the other 12 terabyte drive here so obviously that's the only one i've got selected so basically it's going to move these two shares here and put them onto this disk here now you might find that you can't move all of the data just to one disk you may need to select multiple disks in order to spread the data over those disks if you don't have enough free space on just the one but if you didn't have enough free space obviously unbalance will warn you and so to check everything's going to be fine i'm going to click onto the plan button here what unbalance is doing now is just checking what i want to do is possible to do and it will give me any suggestions of what i need to do in order to make sure it happens correctly now there's one thing to notice here while this is scanning through it's telling us the unbalance needs exclusive access to the disks so we should disable mover and any docker containers that are right into the disk before we run it so i'm going to go back across the unread web ui here and go to the docker tab and i'm going to be extra safe and not only disable the ones that are writing data i'm going to stop all of my docker containers and have nothing running at all basically the less activity on the array is probably going to be quicker to do the transfer okay so that's all of the docker containers stopped and if i wanted to be doubly sure i can just go to the dashboard here and on docker containers switch it from all apps to started only and there's nothing running now my vms i don't really need to worry about i can leave them running i'm going to disable mover so i'm going to go to settings here and scheduler and the mover i'm going to turn it off from daily and i'm going to put on monthly on the first day of the month and today it's the middle of january so i know move is definitely not going to run while i'm balances so that's good i'm going to click on to apply and done so now i know there's going to be nothing being written to the array and nothing else really using it so let's go back to the unbalanced web ui and wait for unbalance to finish the checks okay so if you can see here there's some permission issues on some of the files and folders and it recommends we use the docker safe new permissions tool which is part of the fixed common problems plugin so before i go any further i'm going to run the fix permissions tool so i'm going to go back across to the unraid server and i'm going to go to tools here and as you can see at the moment there's only this tool here called new permissions which is not the one i want to use i want to use one called docker safe permissions so in order to have that i need to install the fixed common problems plugin so i'm going to go across to apps here and install fixed common problems so i'm going to click on to install now this is a very useful plugin for a lot of things but now this is installed if i go back to the tools tab here here we've got docker safe new permissions so i'm going to click onto that i'm going to check this box here now i don't have any pre-clear in progress so it's safe for me to run it and i'm going to click start so this might take a while depending on the size of the array so i'm going to pause the video here and come back when it's done okay so with that done i'm going to click onto well done and go back to the unbalanced web ui you can see here that dry run is checked now dry run will just go through everything and check everything's correct basically do everything but without doing everything so we need to uncheck that tick box and i've got the option here to either move the data or to copy it now i actually want to move the data from this drive onto this drive here i don't want to copy it but maybe you'd want to copy it if you're taking this drive out and he still wanted to have the data here but only have it copied to the other drive but like i said i'm going to click move to move my data and we just sit back have a coffee and relax and it says it's going to take about 39 minutes so let's come back when it's done okay so unbalance is finished it took a little bit longer 1 hour 19 minutes so if i go back to the un-raid web ui here and i look on disk 2 it's still got two folders inside the folders obviously there's nothing there okay so what i'm going to do now is i'm going to shut down the server and i'm going to remove the disk 2 here and with the disk removed and the server restarted up as expected it says there's no disk and there's a disk missing here now it's a good idea probably to take a picture of this screen to take a screenshot showing where all your disks used to be the order of what the disks were especially the parity then next we want to head on to the tools tab here and to click the button saying new config now see here where it says preserve current assignments at the moment it says none by clicking on the drop down arrow we've got the option to keep certain assignments of certain disk types in the array i'm choosing here to keep the pool slots because i want to make sure that all my cash drives are assigned as they were before now in earlier versions of unraid we were able to preserve the parity slots as well this doesn't seem to be in unraid anymore so that's why it's a good idea to have a photograph of how your array was assigned because we want to know which disk was parity anyway with the pool slots ticked i'm going to click onto apply and done and i'm going to go back to the main tab here and we can see there's nothing assigned except the pool slots here both cache drives are there so i'm going to scroll back up to the top and then assign the disks so i'm going to take a look at my screenshot while i do this and first to set the parity disk which ends in z5 now because we're going to rebuild parity after doing this it doesn't matter which order the other discs go in obviously you could make them in size order if you wanted to be neat and tidy i'm not doing that because eventually i'm going to replace this 12 terabyte drive with another 16 anyway right so once we double check that our disk assignments are right we can then scroll down to the bottom of the page and start up the array now on the parity disk you can see that it says all the data is going to be overwritten when the array starts so it's very important to make sure that that's not one of our data disks so one way we can double check is to unassign the parity disk and leave all the other ones assigned then just scroll down to the bottom of the page and start up the array so when the array started we should see all of our data every disk should be available and have data on it but also if you've got the unassigned devices plugin installed then you should be able to see the parity disk listed there and it shouldn't say it's got a file system because obviously it's the parity disk so now with that checked i'm going to scroll down stop the array and then i'm going to add the parity disk back and restart up the array and obviously with the parity disk assigned it's going to start a parity sync when the array is started back up so i'm going to click start to do that so now the array started back up we can see that a parity sync's going on now it's quite a stressful time for the array while a parity sync's going every disk is being read and the parity disk is being written to now if we scroll down to the bottom here because my disks are quite large 16 terabyte drives it says that the parity check is going to take one day five hours so for just over a day should a disc fail in my array i'm going to lose the data on that drive but hey space invader likes living on the edge anyway and who needs bungee jumps when you can just be stressed about your array you get the same adrenaline buzz no but in all seriousness i'm not worried about it really all of my drivers are pretty new so everything will be fine i'm sure now don't forget that we disabled mover earlier so don't forget to come back and actually re-enable it else it's not going to be running now obviously if mover runs during the parity sink the parity sync will probably take a little bit longer so some people might think it's better to wait till the parity sync has finished before coming and re-enabling that but i'm going to do it now anyway okay so now it's just a matter of sitting back and waiting for that to finish my array size now is 60 terabytes and i still got 25 terabytes free so that's plenty of space should last me a while okay so whilst the parity is rebuilding there's one other thing you might need to look at if i go to my shares here for these various shares for me i've got them set to be on various different disks and you can see in the comments here i've put which disks these shares are on if i look at my movie share here it says my movies are on disk five and if i click onto the share here because there isn't a list five anymore it's just set to use all of the disks so i'm going to go back to the main tab here and have a look at the first drive here this one's tv shows so it's not this one and the second drive okay so this is my movies so i need to set this share to use disk 2. so i'm going to change in the comments so it's correct and set the included disks to be only disk 2 and click apply ok so this is something you may or may not have to do depending on whether you set your shares to use specific disks or not i have so i'm going to have to go through and check all of these you may not have to but it's just something to keep in mind okay so here we are the parity sync's been running for 22 hours and we've just got two or three minutes left to finish okay so i can breathe a sigh of relief now the parity sink's finished taking 22 hours to complete and we can see i've shrunk my array to 60 terabytes but i've still got 25 terabytes free now i did this a few weeks ago and if you look at my array now you can see i swapped out that 12 terabyte drive for a 16 terabyte and there's about 700 gigs written to that drive at the moment the array size is 64 terabytes 36 terabytes used and 27 terabytes free also in this server you'll see that there's three zfs pools as well but if you're wondering what i actually did with the other 12 terabyte drives well i made a new server a backup server the main array consisting of three 12 terabyte drives giving 36 terabytes i use that to backup all of my media onto the server also has a single disk cfs pool as well 16 terabytes used for some other backups now yeah you'll notice on this server there's no parity so no disk redundancy at all this server being a backup server it only automatically turns on when it's scheduled to do a backup so i really don't see the point in having any disc redundancy anyway enough about my servers now let's move on to the next method about how to remove a disk from the unraid server without breaking parity okay so over on this server there's just a small array consisting of one terabyte drives so i want to remove this one here disk two but this time i'm going to do it without having to rebuild the parity so just as before i'm gonna have to move the data off here and put it onto the other disks so again i'm going to use unbalance now the process is just the same as in the first part of the video so i'm not going to show it again i'm just going to speed through this part okay so that's finished the data's moved now there's one thing we have to make sure is there's absolutely nothing on the disk at all now at the moment we can still see there's a folder here that had the work share but there's nothing actually inside of that at all now even though i just ran on balance it did leave this empty folder here so i could just manually delete it using command line or i could use a tool like crusader to delete it but running on balance a second time will normally clean up and he left behind empty folders from the first time when it ran so i'm going to quickly run on balance again just to clean that up because like i said it's super important that there's absolutely nothing on the disk at all so with that run the second time there shouldn't be anything on the disk let's check okay so there's no objects not directories and naught files that's exactly what we want now the next thing i'm going to do is i'm going to go to settings here and then on to disk settings and i'm going to scroll down here to tunable mdright method and change that from auto to reconstruct right that's because doing this when we zero the drive it can actually speed up the process quite considerably so to zero the disc we're going to use a script but the script requires us to do a preparation step before running it and that is to create a specially named folder in the root of the disk with nothing else there and to do that i'm going to open up the terminal window and so i'm going to type the command to make a new directory which is mkdir and then a space and then i'm going to put the location of the disk and i can see the location of the disk here forward slash mnt forward slash disk 2. so that's what i'm going to type in now so after disk 2 i'm going to put forward slash and then the name of the special folder which must be in lowercase exactly like this clear hyphen me now having this folder created is a precautionary measure that the script needs because the script will only zero a drive with nothing in it at all except this special folder with this exact name okay so that's done i'm going to close the terminal window and i'm going to refresh this page and there we can see the folder called clear me with nothing else there on disk 2. okay so the next thing we want to do is make sure nothing at all is using the array so i'm going to go across the settings here and go to docker and i'm going to disable the docker service which will stop all of the containers and also i'm going to do the same to the vm service and disable that as well so now we can be sure there's nothing actually running on the server that's using the array but we also need to make sure that any of the computers connected to the network that they don't write to any of the shares as well so during this process make sure nothing is being used on the array at all so to zero the drive we're going to be using a script and we're going to run that script through a plugin called user scripts so we want to go to the apps tab here and do a search for user script and if the plugin isn't already installed on your server then install the plugin and with the user scripts installed i'm going to go to settings here and under user utilities at the bottom i'm going to click onto user scripts now you can see i've got a few scripts here from when i've had this plugin installed before and i'm going to click onto new script and i'm going to call this clear drive and then click on the ok button now we'll see our script listed here now we want to click on the cog next to the script's name and then click onto edit script so obviously at the moment this is a blank script and we're going to copy and paste the script into here and to do that i'm going to open a browser and then in the description of this video you'll see a url copy and paste that in which will bring us onto a post on your own raid forums where we'll find this excellent script by rob j and then we need to select everything here in this script and then copy it to the clipboard then go back to the un-raid web ui and i'm going to delete out the top bin bash here and then paste what i copied from the unraid forums into this window okay so with that done i'm going to click save changes and so now what the script's going to do is it's going to zero the drive it's going to look for a drive which only has this folder on it and nothing else if it doesn't see this it won't run or if it sees the folder and other files on there it won't run either so let's go back to the user scripts and then i'm going to run the script i'm going to run the script in the background that way i don't have to leave the window open when i'm running the script so i'm going to click on here and it says script running so i'm going to click onto done and now if i want to see what's happening with the script i'm just going to click on show log here and we can see here it says it's found an empty disk to clear disk 2 which is correct and basically we have 60 seconds to cancel the script if this is wrong and if we're going to cancel the script we can't just click the red x because it's running in the background we'd have to close this and then click here abort script okay so now we can see it's on mounting disk 2 and now it's zeroing the disk now until the script's finished we don't want to do anything with our server at all now it's very important we don't try to access the disk at all and on the main page we can see there are writes happening to disk 2 and the parity because as this disk is being zeroed the parity is being altered because obviously as the disc is being zeroed this is also being reflected in the parity so now basically we just have to wait for the script to finish and we'll know it's finished when we don't see well that it's running and we can keep coming back and having a look at the log if we want to now obviously depending on the size of the drive will depend how long the script will take to run and now you'll notice here that it says the size of my disk is 8.3 gigs now any kind of weird errors like this totally ignore them this often happens don't worry about it and wait for the script to finish okay so we can see the scripts finish running so i'm going to click on here to look at the logs and in the log we can see this error here which is perfectly normal saying no space left on device and we can see that's confirmed telling us this is totally expected so the drive's now cleared so i'm going to close this log go back to the main tab here now don't remove the disk from the server just yet we're going to leave it in here we just want to stop the array now it's worth taking a photo of the orders of the disks and once you've done that go to tools then new configuration and under preserve current assignments click onto that and click all and close check this tick box here yes i want to do this and click apply and done now i'm going to go back to the main tab and the disks are still in the same order here but now with the disk 2 i'm going to unassign it so now i'm going to scroll down here and it's very important here to check this box here saying parity is already valid and i'm going to click start and so now we can see disk 2 is missing we've only got parity disk 1 and disk 3. and this 2 is here as an unassigned device so now i'm going to shut down the server and just remove this disk fully okay so now the server started back up there's only these three array disks installed the parity and two data disks and obviously the cache so i'm going to start up the array and everything looks good so you might be thinking well how do we actually know the parity is actually correct well i guarantee it is and if you see my earlier video on parity then you'll know why but hey don't just take my word for it now if we run a parity check now if this hasn't worked then we're going to get parity errors because it will correct them but we'll run a parity check and we won't get any errors but let's just do that and see what happens so because the array is really small it says it's only going to take an hour to complete so let's fast forward and come back and see if there are any errors okay so the parity check's finished and we've got no sync errors that proves that the parity was valid before so using that script and removing a drive is absolutely fine so i guess that brings us to the end of this video now we've seen two methods to shrink the unraid array now i really hope you like this video enough to give it a thumbs up and share it with your friends and if you're not subscribed to the channel then why not subscribe anyway i just want to give a huge thank you to all of my patrons and supporters out there thank you so much guys for enabling me to make these videos and if you want to join those great bunch of people and help support the channel then please see the links in the description below anyway guys this video is getting very very long so it's time for me to go but whatever you're up to for the rest of the day i hope it's good and i'll catch you in the next video

Channel: Spaceinvader One

Views: 47,417

Keywords:

Id: nV5snitWrBk

Length: 24min 51sec (1491 seconds)

Published: Tue Feb 08 2022