Windows 10 Storage Spaces: Use ReFS, build resiliency and data protection

[Music] [Music] [Applause] dad [Music] hello and my name is lowell vanderpool and this channel is dedicated to it professionals i.t students and anyone who's interested in technical subjects windows 10 storage spaces is an incredible technology put inside of a client operating system so we're going to look at data center storage for the common man one of the goals of this video is to give you a very practical guidance on anis and a demonstration on how to set up enterprise grade storage system that's resilient and inexpensive even though i'm going to show you an enterprise grade storage system that most people can afford and set up you still have to have a carefully designed and tested backup plan windows 10 includes windows 7 backup which can comprehensively backup your data and provide bare metal recovery don't forget you have to have backups i can't tell you how many times i've seen small businesses and home businesses that depend entirely upon say lots of 4k and or 8k video content photographers that use high resolution photos semi-professional musicians who have tremendous investment in their audio recordings or any small business owner with critical business data any one of these businesses if they lose their data they have lost their business period and yet many times it's the last thing they consider is the fragility of their data one incident of ransomware and they would be finished windows 10 comes with four enterprise technologies hyper-v which is a full enterprise grade virtualization product and it's built into a client operating system amazing storage spaces and enterprise grade storage system raf file system and the linux subsystem these are full blown enterprise technologies that are found in your copies of windows 10. let's begin by looking at refs this is a new file system its code name was protogon the resilient file system or refs was released in 2012 with server 2012. the technology now is included in windows 10 but it can only be used as part of storage spaces refs is a resilient file system used for enterprise storage ntfs is a good solid file system but refs has a high data protection from corruption it uses what is known as the b plus trees for all data disk structures it uses integrity streams so that all metadata has 64-bit checksums and these checksums can also be used for data it does not need error checking tools like check disk it also uses a feature called copy on write which protects your data should you lose power or you have a system crash other modern file systems like zfs btrfs for linux as well as the apple file system also use this copy on write feature so why do we use a b plus tree storage system the primary value of b plus tree storage structures is efficient retrieval which is perfect for a file system many popular file systems and databases use b-plus tree linux extended four uses it for indexing microsoft sql database oracle sqlite the be file system for bos journal file system used by aix and hpux it was described in 1972 by rudolph baer and edward mcrae raf has some advantages right out of the box over ntfs it scales better for large files and volumes it can handle up to 35 petabytes if you use 64-bit clusters and 16 petabytes if you use 4k clusters in almost all cases you and i the common man will use 4k clusters max file size 35 petabytes if you use 64-bit clusters 16 petabytes if you use 4k clusters if you're using normal data 4k clusters is preferred if you're using a lot of say 4k video or ak video then 64-bit clusters might be preferred the number of files stored in a folder up to 18 trillion our efs is compatible with a lot of existing apis and technologies it still supports access control lists bitlocker mount points volume snapshots and on and on in this list disk quotas etc it does support things like data dedupe refs does not support compression page file support bootable disks 8.3 file name support and you can't just format a hard drive you need storage spaces and if you convert an ntfs to ref you're going to lose the data refs and ntfs are not the same it refs uses 64-bit addressing and the main structural element in this file system is the directory again cluster sizes range from 4k which is typical to 64k now don't get confused storage spaces has nothing to do with re fs now rafs is simply a file system and it can and only can be used with storage spaces you still can use ntfs with storage spaces windows 10 provides two disk management systems storage spaces and the old disk management even with windows 10 you can't avoid disk management it's still critical to manage bootable disks it contains the ntfs partition for the operating system and it still needs to manage the page file so disk management is not going away anytime soon so let's talk about the fundamentals of storage spaces number one the the word pool this simply means a set of physical disks the next term is called a slab it's a chunk of disk space 250 megabytes all physical disks are sliced into what is known as slabs and the last concept is space it's just basically a virtual disk that's created from a set of slabs now there's three characteristics of every storage space one you can have simple layout mirror three-way mirror or parity you can also use a fixed or a thinly provisioned type of space and remember when we say space think of virtual disk and for that virtual disk or space we can use a file system of refs or ntfs some concepts to remember about windows 10 storage spaces is it's basically software defined storage it's expandable storage you simply add more drives and it grows now you can have either two-way mirror which acts like a raid one you have to have a minimum of two disk or parity which is like raid five you have to have a minimum of three discs you can have three-way mirror which requires five discs or you can have simple it's kind of like raid zero with a minimum of one disk remember storage space is the only place you can use refs but you still can use ntfs on storage spaces if you so desire now windows 10 storage spaces is designed to use internal disks so it can use sata spindle disks that are internal internal ssds m.2s internal sas disks i'm going to teach you how to use external if we're going to use windows 10 storage spaces for resilient redundant internal storage your limitation is how many sata ports you have on your motherboard it's easy to implement you plug in a cable and a hard drive and off you go expandable based on how many sata ports you have on the motherboard so think about that and it is the fastest transfer rates because you're taking full advantage of sata or if you have nvme you're getting the nvme speeds windows 10 storage spaces fully supports internal hard drives if you want to use windows 10 storage spaces for external storage you can do this also you can buy a variety of external enclosures most of them connect via network which i encourage you don't use usb 3.0 or 3.1 generation 2 is the fastest you can set up all your disks as just a bunch of disks do not use the raid software built into the enclosure we want to use storage spaces not their raid this is the best way of building a highly reliable resilient backup system external enclosures are very very desirable especially with storage spaces even though by default storage spaces will not use them without some reconfiguration which i'm going to show you make sure that your enclosure has adequate power supply make sure it has hard drive cooling these are great because you can add hard drives easily and replace defective hard drives and if you need to you can pull the hard drives out put them in another enclosure they usually provide massive storage capacity and they make for fantastic backup systems one thing that i really like about external enclosures is that i can disconnect them except when i do a backup so i'm protected against ransomware now some of the disadvantages are it uses usb 3.0 so it's got a little bit slower transfer rate if you can get the usb 3.1 gen 2 which is 10 megabits it's pretty good again the easiest way to use storage spaces is internal drives so they'll work with almost all internal disks make sure your power supply on your pc is oversized to handle that spindle drive startup make sure it's got good fan support so that you can cool those hard drives anything over three terabytes you need a lot of cooling on those disks it makes it more difficult to remove disks and add them and expand them but it's doable when you're purchasing disks for storage space always buy enterprise sata hard drives enterprise disks have a much more robust mechanical system and better bearings the mechanical systems just last longer they're more expensive they typically have a five-year warranty you can even buy enterprise used hard drives on amazon with a three-year warranty if you need if your power supply is low and you can't you can't upgrade your power supply on your case you can always go down to two and a half inch enterprise hard drives they run cooler and require less power as of the time that i did this video there were only a few companies that were using usb 3.1 generation 2 usb if you can get it it's worth it it's 10 gigabits it really makes a difference in transfer rate if you decide to go with ssds for your storage space use them only internally so that you get all the maximum transfer rate that you can shop carefully and purchase maximum capacity for your budget the great site to check out disk prices is the one listed in the url it's relatively painless to force storage spaces to work with usb externals i'm going to show you how i'm going to walk you through the process of how to get storage spaces to work with external hard drives which typically it doesn't want to do now this is simply two two and a half inch spindle drives one's a toshiba ones of seagate they're very small 500 gigs and i'm just using them as a demonstration i've got a little usb 2 port toaster and i can plug in three and a half inch hard disks or two and a half inch hard disk or sata drives also this works really really well i've used them for production servers i've used it in my in my own use because i do a lot of video a lot of photos and i need a reliable backup and storage mass storage so i'm going to demonstrate how to get this to work and walk you through it step by step so let's begin by opening up computer management console and you can see my two usb disks and right now they're unallocated they're simply plugged in i'm going to go to device manager and you can see i've got their volume names and i've changed the volume names of both disks so it indicates to me their usb that way when i'm working with disks i don't make mistakes the volume helps me identify what that disk is in this case i could use the word external in the case or i could use usb disk or device so this just gives me a heads up because you can see i have a lot of disks here i don't want to make a mistake and blow away a good disk i've changed the volume names of my usb drives so that i know which disk i'm dealing with so that's something you want to remember so i look in device manager they're all there go in disk management and i can see that my two usb disks are unallocated and unused if i launch storage spaces right now and i try to use these two disks i'm going to create a new pool remember creating a new pool is about gathering a couple discs together and i'm going to use here i'm going to use disk 5 and disk 6 and notice there's my two disks and i'm going to say we'll go ahead and create a pool now storage spaces does not like to use usb external disk and you can see it failed it's not going to do it so we have to do some things to these two disks so that we can convince storage space to use them so let's begin with step one so the first thing i'm going to do is come to my first usb disk and i'm going to come over and in disk management i'm going to click properties and i'm going to go to the policy tab usb drives usb dip flash drives all want the ability to be able to quickly remove them you can just pull them out plug them in and that's the way we want but one step that we're going to have to take is to enable write cache in windows now when you do this you can no longer unplug this drive like you did in the past you're going to have to always use the safely remove hardware feature in windows whenever you want to remove these disks now i'm going to change this policy this is step one so i'm going to do this for disk 5 and i'm going to come down here and do this for disk 6. so i'm going to do this for both disks so that's the first thing i want to do is go into each disk and change this quick remove policy to enable write caching and we'll say okay now i'm not done i that's my first step the next thing we want to do is we want to tell the usb hard drive that it is not a removable disk so we're going to use a tool from lexar which is a flash drive manufacturer it's a usb format tool and make sure you go to a reputable site to get this utility in this case i'm going to pendrive apps.com which has a wonderful selection of portable apps and if you've watched my videos you know i love portable apps and you can use this tool and the only thing we're going to use is the flip removable bit feature so we're going to use that feature now i've downloaded the tool and i've got it right here i'm just going to execute it so before i run the the flip bit feature tool i'm going to come over here create a simple volume just walk through and give it a drive letter and we'll do the same for the other we're just going to put it into disk management so it will be seen all right it's done now let's go to the utility and notice that it sees dne as removable usb drives or they could be flash drives and i'm going to choose d first and i'm going to flip the bit and it's going to change a structure on the disk that is now going to tell windows this is a non-removable drive then i'm going to go to e i'm going to select e and i'm going to flip that bit and then i'm going to exit i'm done that's all i need to do so now we finish that let's go back to storage spaces and i'm going to create a new pool which basically is allowing us to organize some disks and i'm going to come down to disk 5 and 6 and remember before this wouldn't work we're going to create a pool and voila it worked so that's all i had to do to take those usb externals and make them available to storage spaces so now we're ready to go so let's take a look i'm going to i'm going to choose a drive letter that is not the next available drive letter and never use an available drive letter i'm going to use one like r further down in the drive letter assignment we're going to use refs you always want to use refs the whole idea is you're gaining resiliency with the type of raid capability that storage space gives you there wouldn't really be a value of just using ntfs when you can use a better file system i'm going to you you can see it we can use two-way mirror i've got two 500 gig which gives me about a terabyte but if i mirror i'm going to get about 500 gigs of space now i can say i'm going to let's say i put say 20 terabytes that doesn't mean i have 20 terabytes it means that i can grow i can keep adding drives up to 20 terabytes i can change this to simple i can change this to three-way which i don't think i can because i don't have five drives in most cases the most people have recommended not to use the parody it seems to slow things down especially if they're usb hard drives either the two-way mirror which is probably the best in my case so i can go ahead and create a storage space and i'm going to use a letter r rafs and go ahead and create the storage space it's going to format the drive and notice it puts it in my explorer view so i can see now our drive and its volume label is storage spaces if i come down here in disk disk management it sees it as one disk not two and it is now drive letter r and it is refs so i can see all of that so let's take a look at the recovery ability the resilience to failure so i've got my storage spaces our drive and i put some material in there you can see i've got about four gigs of material in this new storage space volume and i'm going to go ahead and click ok so here i've actually taken one of the hard drives out of the array shut down my computer popped it out and you can see in storage spaces it does see this problem it sees we've definitely got a problem with the storage spaces array one drive is there one drive is in this case totally gone and yet i can come up to my explorer and there's my r drive and all my tools all my files are there it does provide us all the necessary redundancy and mirroring the raid capability that we want even with completely failed hard drives this is the beauty of storage spaces to help lower my cost for hard drives i went for used enterprise sata hard drives i was able to substantially lower the cost of all the hard drives that i use make sure that when you're dealing with storage spaces purchase the same make size and model drive keep all the drives the same and it's wisdom to buy an extra spare disk while you're buying disks for your array go ahead and buy a spare disk look at your return policies i i would encourage you to avoid ebay pay attention on how the drives are packed and shipped if they're not shipped like new drives i probably would be suspect make sure you test the drives upon receiving with with a tool like crystal disk info it's a great tool to test the disk if you can burn at least 24 hour burn-in for all your used drives i purchased four used drives and you can see i ran crystal disks on all of them and based on the power on count they had never been turned on so they were brand new they were sold as used and i was able to save probably seventy percent on the price so these were very nice enterprise three terabyte drives a couple points about uh the disc enclosures if you decide to go that way you can either get daz enclosures or nas enclosures direct attack storage are explo enclosures in which you attach them to your pc via usb esata firewire thunderbolt those are all das or direct attached storage you'll usually get better performance with a das enclosure if you're going to connect via network cable that's usually going to be a nas device there's benefits to nas and there's drawbacks to nas generally the lower price nasa's are very slow so that's one drawback daz and nas can have raid built in but if you're going to use storage spaces you don't need it so you can save some dollars and just buy an enclosure that supports as many disks as you want and storage spaces is going to provide you the raid if you're going to use usb externals make sure that you disable the usb suspend setting so that you're not going into power mode and turning off your usb external you want to go into each disk that you use for a usb external and make sure that you disable the quick removal feature and turn on the right back caching you also want to download the boot it utility and flip the bit so that it is no longer seen as an external device you can see once you have your storage spaces running you can actually see your storage spaces in your device manager also in the video description you'll have a link to download the notes for this video this will have extensive step by step on how to do everything that i'm showing in this video in the notes i'm going to include a lot of powershell commandlets that you can use to look at your your storage spaces and repair them i have an entire section in the notes based on when things go wrong with storage spaces so i've got a lot of powershell and troubleshooting ideas on how to fix your storage spaces should something go wrong [Music] [Music] you

Channel: TechsavvyProductions

Views: 3,999

Keywords: Storage Spaces, FeFS, ZFS, BtrFS, B+ trees, integrity-streams, 64-bit checksums, volume snapshots, disk quotas, block cloning for VMs, EFS, pool, slab, space

Id: 4xdVcWvtUnc

Length: 23min 15sec (1395 seconds)

Published: Mon Nov 23 2020