How to Achieve a Petabyte of Storage

hey guys brette kelly here iron D engineer at 45 drives now we make large storage service as you can see our store a native storage pot now what I want to show you guys today is how to take these things put them into a single namespace cluster to create one giant storage pool so why talk about this now clustering itself is nothing really new but over the last few years two things have really changed software used to be very complex and expensive to run a cluster which is now evolved to the point where kind of no cost you can put cluster together in less than 10 minutes two is the hardware performance has been going up but the costs have been falling thanks to these changes clustering has really become a very practical and cost-effective way of solving your storage challenges and actually without further ado let's let's get into it let's go into the lab so what I want to show off here is our low-cost petabyte solution so in this configuration I've got three s forty fives all strung together full terabyte western gold drives to give us a whopping 1.1 useable petabytes of storage so why a petabyte well we just decided because it's a really cool number one petabyte thousand terabytes why not it's not everyday you can put together something that could hold a petabyte of data it's really cool that it can be done with just three s45 lights all full eight terabyte drives for a total of 83 thousand dollars but what's really amazing about all this is I can put the whole thing together not including physical racking and network configuration in less than 10 minutes and it costs you less than 100 grand one of the systems well the one we built this one with but one of the systems we like to work with glossary fest a clustered file system allows for multiple units to work together to keep a highly available and a really powerful storage solution so not only is glossary fest incredibly powerful oh really easy to scale it's open source and it's relatively easy to set up so because of these reasons the biggest players in the IT world like Facebook and Red Hat and many others are building their entire IT infrastructure around this system it really is something special so what can you do with a cluster cluster essentially you can do anything that you could do with a traditional server but so much more like massive scalability high availability for 24/7 uptime and just endless performance it can interface with all your company's current IT infrastructure including by using Samba NFS I scuzzy Hadoop Amazon AWS or Swift stack it's really incredibly versatile it's just a large cluster volume that can play with anything really so it's low cost it's incredibly versatile but can it perform so I'm just going to rip a four gigabyte file off my cluster my petabyte cluster onto client here the old transfer comes up oh yeah pretty fast so with a cluster like this it will scale linearly in capacity as you add drives and it scales linearly with performance as you add servers so what I'm trying to say is you will never replace this solution you will only ever grow it alright so we told you what it could do we shows you how fast it can go now let's just break down the price real quick so in this configuration I'm using three s 45 enhanced each around you about six to $300 which each one filled with forty five eight terabyte Westin WD gold drives now these will cost you about five hundred forty dollars a piece but these are the gold standard hard drives they have a five-year warranty and the reason the warranty is this long is because they are incredibly unlikely to fail so the tally goes up bring all the hard drives in brings my petabyte cluster I built here under a hundred grand and one more thing this whole solution is built on top of Sint OS meaning that you get the incredible openness and versatility of a full Enterprise Linux system oh I forgot annual software licensing fees are zero because this is an open open source solution so pretty awesome so to sum up you get a full high-capacity enterprise storage solution for under under a hundred thousand dollars in fact you can get the servers for as low as seventy five dollars per hard drive slot meaning this whole petabyte solution can come in as low as eighty three thousand dollars it's got high performance it's really easy to scale it's got the ability for high availability it's pretty easy to set up and it's incredibly versatile so it just fits right in and works with any of your existing IT infrastructure so there you go that's what we can do in just under ten minutes a full enterprise class clustered storage solution for under 100 grand thanks for watching and as always if you have any questions feel free to email us at info at 445 drives comm or drop a question in the comment section below thanks guys you you you you

Channel: 45Drives

Views: 122,987

Rating: 4.8353791 out of 5

Keywords: 45 drives, clustering, glusterfs, cluster, big data, large storage servers, large storage server, petabyte, clustered storage, clustered file system

Id: C2uLSOmRx_c

Length: 5min 57sec (357 seconds)

Published: Mon Nov 28 2016