How and why to PreClear your hard drives before adding to your array

well hi there guys and welcome to this tutorial this tutorial is a quick guide which will show you how to prepare a hard drive before adding it to the array on your unread server and we'll do this by using the pre-clear plugin to thoroughly test out the hard drive group so guys in this video we're going to talk about pre clearing our disks in done raid and why we should do that and also obviously I'm going to show you how to do that so before we pre clear our disk let's talk about why exactly we need to pre-clear our disks and what happens during the pre clear process so to start with why do we want to pre clear our disks well I think the most important reason is is it helps check against a drive failing due to early drive mortality now new disks are not tested extensively at the factory when manufactured you know maybe some enterprise class drives are but certainly not the normal commercial drives that you and I would probably buy and when a hard drive is new and it's not being used this is the most likely time for a disk to fail hard drive failures follow what's called a bathtub curve meaning that there are more failures at the beginning and at the end of the expected life than in the middle and the failures at the beginning are basically called early drive mortality and by running the pre clear test we're able to check to see if our hard drive is likely to fail during this period of time and basically the way pre-clear does this is it reads every byte on the hard drive and then it writes a 0 to every byte on the hard drive and then it reads back every byte it's just written and all the time while simultaneously performing background operations such as generating random head movements and it also compares the smart values for the drive at the beginning and the end of each pass so you can see that pre clear is performing a very thorough test of the hard drive so that's the main reason for pre-clearing a hard drive but also when expanding your array it saves time when you're adding a new drive to the array that's already parity protected if you don't run a pretty clear when you add a new drive to the array and raid must clear the drive first and during this time you can't actually use the array at all and the amount of time it takes is roughly three to four hours per terabyte of capacity on the hard drive now adding a drive to the array and increasing the size of the array from an added drive is different to actually replacing a drive that's already in your array with another drive when you're adding a drive to an existing parity protected array there's no guarantee that the drive has nothing but zeros already written to it and if there was anything other than zero is written to the drive then the parity would not be valid so in order to keep parity valid and rate must 0 or clear this disk in order to keep everything right however if you have done a pre-clear previously on this disk there is a signature that's written to the disk a none raid will check for this and if it sees this signature present then it doesn't need to clear the disk and so then adding this disk it will only take a couple of minutes to expand the array and have the array fully working and ready to use however adding a new drive to an existing array that's actually replacing an existing drive there is slightly different when a drive is replaced by another drive an array doesn't need to zero or clear that drive that's because it's going to do a rebuild process when the parity and the data drives and writes every byte on that drive anyway so it really doesn't matter if the drive was zeroed or not before ok so now you know a little bit about pre clearing your disk and why to do it let's go ahead and pretty clear a disk now the first thing we're going to need to do is to go to our community applications as normal and then type in is all one word pre clear and then click on install to install the pre-clear disk plug-in okay so now with our plugin installed we can pre-clear disks so you need to attach your disk to the server to a spare SATA port and obviously the disk can't be already part of the array as you can see here I've got a 3 terabyte hard drive as an unassigned device and this is the drive that I want to pre-clear now before I'm going to pre-clear it I'm just going to have a quick look at the smart information just to check there's nothing obvious there it's going to tell me that my drive is already bad ok so the smart information looks good so now we can go ahead and pre clear this disk so click on to plugins and then on to pre-clear disk and there you'll see my 3 terabyte hard drive is listed there and all we need to do is just click on to start pre-clear and that brings us to the options for the plug-in you can choose which script to use what operation you want to use how many cycles and whether it notifies you by email etc and also you can choose to skip the pre read and skip the post read doing that you could actually just zero the disk if you chose to skip both of those that we're not going to we're going to do a full pre clear of the disk so then just click on to start and pre-clear will start the first stage which is the pre read and at the top here you can see the status of the operation things such as the read speed of the data now it does take a long time to pre-clear a disk for me on this three terabyte one it took six hours per stage so a total of about 18 hours altogether so I'll this pre clears running let's just talk about in a little more detail what each phase does so this phase the pre read phase is it's reading all of the bytes on the drive and if there's a bad sector the initial read allows the smart system to detect it however it won't remap this after the read but what it will do is remember that this is a weak sector by marking is a depending reallocation because smart doesn't know whether this sector contains important data smart will not reallocate a sector on a read unless it's able to successfully read the data after several retries and at this point it can then remap the sector and write the data elsewhere without risking losing any data okay so there we have the pre read phase finished and now we're on to the write phase now the write phase zeroes the disk this writing stage will cause the smart subsystem to remap any sectors that have been marked pending reallocation during the pre read phase since this is a right there's no need to read or actually know the contents of that sector because it's about to be overwritten anyway smart will then try actually one last time to access the pending sector to confirm it's actually bad and then it performs a remap of that sector writing the data to the newly map sector actually very rarely the sector does respond and smart will unflagging as a pending sector and does not remap it elsewhere so with the right phase finished we're now onto the final stage and this is the post read phase this step confirms that all sectors just written are actually in fact full of zeros now two things this can pick up well obviously non-zero data after having just filled up the disk with zeros would definitely mean the disk is not good just as it did in the pre read phase the post read phase is also engaging the smart subsystem if the post read identifies additional sectors that are problematic it will mark them as pending again this would indicate that the drive is bad because the first read didn't see these sectors as a problem but now they're being seen as a problem so the drive is obviously failing okay so finally the pre-clear disk has come to an end and on my browser notification it says that my disk has passed so let's now have a look at the pre-clear report for that we're going to need to go to the main tab on R & R a GUI and then click on the blue symbol next to my hard drive ad at the top you can see the various stages during the test and how long each one took and whether it succeeded or not and then the second thing you can see is the smart status and you can see things such as the power on hours or how long it was on for my MS on for 18 hours and you can also see how hot the drive got during the test might only went up by 4 degrees so I'm pretty happy with that and it's just one error I have got is my CRC error count actually went up by one this isn't really a serious error it just means that somewhere the data didn't get through quite properly this can often be in to do with electrical interference or it can be in to do the faulty cable so what I'll probably do is run this test again after having changed the cable and see what happens a lot of people always wonder like how many times you to run the pre-clear well I think if it dis makes it through the pre read the write and the post read with no reallocated sectors no pending sectors and verifying the zeros were all written correctly well you can be pretty sure that the drive is good yes or some people prefer to run two or three or sometimes maybe more cycles on a disk and it's up to you how many you want to do but I think one is enough and I'll stick with using one cycle unless I think a disk is suspect in some way if you think about it a lot of us home users may use our array for the storage of a lot of media so this type of file is written once onto the array and doesn't change so hard drives that contain pretty much static data are probably likely to do more writes during an extra two cycles of pre-clear then they may do join their whole lifetime inside your unread server if they were just storing media etc but obviously this is just my personal opinion but in my experience it's been that pre-clear does a fine job with one-pass of sifting out the high percentage of drives that would otherwise experience an early drive failure and so guys that brings us to the end of this video I hope you guys found it interesting and found it useful and if you did guys then the please I'd appreciate it if you hit the like button and if you want to see more videos from me in the future then please don't forget to subscribe anyway guys whatever you're up to for the rest of the day I hope it's good as always I'll catch you in that next video

Channel: Spaceinvader One

Views: 67,662

Keywords: unRAID, unraid, linux, preclear, pre clear, harddrive, hard drive, raid array, array, check harddrive, harddrive failure, harddrive diagnostic, how smart works, smart tests, tutorial, guide, how to, how to preclear a harddrive, how to preclear, preclear script, limetechnology, lime tech, limetech, nas, server, kvm, quemu, plugin, unraid plugin

Id: csGYrd5G0ik

Length: 11min 21sec (681 seconds)

Published: Fri Dec 09 2016

Reddit Comments

[u/spaceinvaderone]

Hi, Guys. Here's a video tutorial about preclearing your hard drives. It gives a little information about how it works and how to use it. I hope some people find it useful :)

[u/ghettosorcerer]

thanks for this u/spaceinvaderone! This is really, really useful!

I'm new to unraid, and I'm curious to see if there's any other hardware tests or general maintenence that you would perform before writing to disks?

Thanks!