Unboxing 3 PETABYTES of storage!!

it's actually kind of amazing how many wires got crossed on this project but we are finally doing it we are building petabyte project number two and not a moment too soon we may not even have enough space left on our servers to offload the footage that we are recording right now and - you might say you could just stop being such a digital hoarder and oh I don't know delete some bloody data but I actually have the perfect counter-argument to that you sound like my wife just let me have my fun and we're gonna have some fun today ladies and gentlemen because I accidentally have over three petabytes of hard drives smartdeploy makes it easy to handle daily IT tasks like windows imaging patching updating apps and migrating user data you can do it all over your existing network or the cloud without leaving your desk get your free offer at smartdeploy dot-com slash Linus [Music] alright so the first problem was entirely my fault actually I told Seagate that our goal was to show off one petabyte of usable space in a single 4u enclosure instead of doing it in two enclosures like we did last time and I told them that to do that I would need 75 of their 16 terabyte hard drives to account for the space that we'd lose to formatting overhead and parity data so that's true in five ZFS raid z2 arrays we would be able to lose up to two drives per needeth so that's up to a maximum of ten of our 75 drives before we would actually lose any data and that would still yield over 950 terabytes of accessible space one small problem though the custom 75 drive chassis that I thought I asked 45 drives for they were like yeah yeah there bud your servers in the male vibe you're welcome bud their Eastern Canadian they really do sound like that it's amazing and I was like so is it the 75 drive custom one and they're like what are you talking about there bud I apparently never clarified I needed 75 days so it has 60 so it looks like we're gonna have about 750 terabytes of usable space but hold on hold on guys the title is not clickbait I am still gonna have one petabyte of raw capacity in here the difference is that we're gonna make up some of that shortfall with solid state all right so let's take a look at the drives that Seagate sent over here so there's actually more boxes here than I expected which is interesting so we're all we're all learning things today this can go here so these are the ones that are right these are the ones that we can do first this is so hilarious oh my god there's so many of them that this is actually like like this will actually build up pretty high I'm not quite sure how it happened but this was error number two Seagate's iron wolf nas drives are designed for network attached storage use respect for a million hours mean time between failure and a hundred and eighty terabytes of access per year they've got a three year warranty and they feature Seagate's agile array a combination of hardware and firmware features that make them perform better in raid arrays they've got RV sensors and better vibration tolerance tuned for performance and reliability and multi Drive arrays and a combination of solid performance and power consumption across a wide variety of workloads including video editing which is our primary concern around here we actually end up editing video directly off of the vault more often than you'd probably think because whether it's a big project and 1x server has no room or because a sponsor wants to change after the fact or whatever the case may be so thank you Seagate appreciate you fam these are great drives and we've recommended them loads of times except for one small problem I really don't know where the communication wires got crossed but these are rated for use and enclosures of up to 8 drives at a time ok then so Seagate then sent over a few boxes of their iron wolf pro drives putting us up to a total of over 2 petabytes of storage but those are also only meant to have up to 24 drives in an enclosure I mean honestly speaking I would have been perfectly comfortable with the iron wolf pros they've got an extra 2 years of warranty compared to the regular iron wolf they've got included data rescue service and they've got a greater rating for both their per year use and mean time between failures but thing is we're supposed to be setting a good example for you guys and when I clarified hey guys so the plan is actually to put all the drives into one system they sent over the big dogs meet the exos 16 in its top current capacity of 16 terabytes each of these is rated for a massive 550 terabytes per year of access and two and a half million hours mean time between failure with all of the vibration sensing and mitigation technology at sea gates disposal to rate them then for an unlimited number of drives per enclosure I have never seen this much storage in one place in my life this is over three raw petabytes of storage 225 drives times 16 terabytes each the bad news is Seagate says that I have to use the earlier shipments of our drives for other stuff or send them back so make sure you're subscribed so you don't miss some more Nazz building collabs with other youtubers that's one of the ideas that I had anyway let's use this opportunity to take a closer look at our enclosure now I am wicked excited about this server so this is an early model this is a prototype of their next generation store inator sorry oh oh now from the outside this looks like a regular old plain store Nader with 45 drives typical all sheet metal construction and all that good stuff but oh that's different they have stepped up their game so they actually went from a cable based backplane system where every single port was individually wired in to these PCB back planes so this dramatically simplifies the cabling since they're just running one quad port SAS cable for each of the four bays and it also should theoretically improve reliability they've also put in some logic to stagger the spin up to the drive so you don't get that same kind of power surge when you first turn on a story nadir and all up to sixty drives are like let's start like ramping up very cool I'm not ready to actually build this thing up yet though because there is another surprise now I don't know how much of this came about because of my request or how much they were working on already but what's this ten forty-five drives has finally joined team red that's right so we've got an AMD epic processor in here I'm actually not a hundred percent sure exactly what the model number is and then it's equipped with what are we looking right here just shy of a hundred and twenty eight gigs of ram now there was a bit of a mishap on our unit and this is like an engineering sample board the gigabyte provided so I couldn't really get a new one two of these memory slots are dead but this is a hard drive based storage system so I'm not actually too worried about the extra couple of channels of memory killing our system performance like it did in our all nvm enas with that said I did allude to needing some SSDs in order to make up the difference in capacity between the 60 drive one and the 75 drive one that I thought 45 drives was working on that is where these come in and I need to actually find out what the devil they are all right here we go so there's a feature of ZFS called adaptive replacement cache or arc and essentially what it does is it takes the most frequently used data from your hard drives and then stores a second copy on your system memory so that you don't have to go all the way out to your spinning disks in order to access it and that's especially important for something like running VMs or a database where a lot of the lookups are going to be to the same few entries we're already using that on the existing petabyte project you know the one with the two bays what we are not using is something called l2 arc or love - Ark thing is sure you can add hundreds of gigabytes or even terabytes of memory to a system now but the cost can be quite prohibitive so that is where SSDs come into play so this right here is a seven point six eight terabyte SSD so nearly eight terabyte SSD from micron that's yeah it's only SATA 6 gigabits per second but even though that's obviously a lot slower than system memory it's much faster than spinning disks and easily enough to saturate our 10 gigabit or even a 40 gigabit network connection and then importantly it is much cheaper than just chucking more RAM into the system now something to bear in mind here is that we ended up putting six of these in here in order to get our raw capacity up to the petabyte that we promised but l2 Ark actually does not scale especially well with a ton of capacity so it's possible that what we'll end up doing is only using some of it cheating a little bit but for the time being hey we got a pata by the capacity also just like Ark it only has a copy of the data on it so it doesn't actually count towards your total capacity but these are just minor details like you know which way the drive goes in for example I know I'm gonna get judged so hard for this but while I could put a dual 10 gigabit mezzanine card in right here in this open gap in the back I have actually decided to use one of our ancient connect X 240 gig InfiniBand cards because I had a slot that didn't have a cover on it and I don't know what else to do with this ancient thing anyway and it's I mean it's 40 gig even though it's an older card it's plenty for hard drives so that's in there with its 3d printed bracket from that video a long time ago this is a really nice feature so I've actually done this the janky way just like hot gluing or double-sided tape enough an on top of my HBA cards so these are the controller cards for all these drives we're going to plug in but hey now there you know an officially sanctioned way to do it sweet so we've got a cooling fan to take care of all of our add-in cards down here very nice Wow now that they've got so much space here they could definitely modify the chassis and do a third fan here if they really wanted to that's freaking awesome this is an example of a really old story inator this is from about five years ago you can see there's a lot of oh there's a lot of things about it that are a lot worse so this horrible horrible mounting system for the drives will flake these rods holding the thing down that wasn't great you can see there's a lot less space in here oh right I haven't even talked about the new mounting system for the drives okay we're gonna do that but first I want you guys to check out this is how they used to cable it up what a nightmare we've actually done a swap for a dead port on one of these things and it was not a lot of fun this right here is apparently a 3d printed like kind of friction mount and then that works with the spring mount but they already had on some of the newer chasse C's along with that PCB backplane which apparently makes it easier to align the slots perfectly so that it's easier to put the drives in and out so let's see yeah that actually worked out so in used to be fine anyway oh okay that's not bad so all that remains now is to install sixty drives about 58 I already did too you can tell this one is very prototype it's got lots of scratches and dings and stuff I believe this really was there working sample of it before they sent it over to me you know you could call it used or you could call it pre tested home stretch and 960 terabytes of raw spitting storage along with 46 terabytes of SSD storage for a total of 1 petabyte in a single chassis but I'm not quite done yet right now the one thing that's missing here is a slog device so I already talked about read caching which doesn't really have any dangers associated with it because you're just making copies of this data to put on your RAM or on your l2 arc but write caching write caching is something you can do with ZFS and you can use your memory for it but the problem is that in the event of a sudden power loss which who knows could happen any in-flight data that's sitting in RAM but hasn't been committed to your hard drives yet will be lost so it might be worthwhile adding something like an octane SSD to one of our PCI Express slots over here to handle caching data that is being written so that we won't lose it in the event that it's sitting in RAM limbo and hasn't been committed to persistent storage all that though is going to be reserved for part two where Anthony and I are gonna team up to get this thing up and running on the network so we can start offloading some of the data from the original vault to the new consolidated vault yes my friends the capacity of two vaults is now one vault technology amazing speaking of amazing technology pulse whey is a real-time remote monitoring and management software that helps you fix problems on the go by sending commands from any mobile device it's compatible with Windows Mac and Linux and pulse way single app gives you remote desktop functionality you can get real-time status system resources logged in users network performance you can manage Windows updates and more with pulse way you can create and deploy custom scripts to automate your IT tasks and you can scan install things update all your systems on the go it's super cool and super powerful and you can try it for free at pulse wave comm or through our link in the video description so thanks for watching guys if you're looking for something else to watch why not check out the epic saga that was our nvme storage server upgrade yeah we're basically replacing the whole server room right now if you guys didn't sort of pick up on that

Channel: Linus Tech Tips

Views: 3,273,115

Rating: 4.9297519 out of 5

Keywords: seagate, exos, ironwolf, pro, hdd, hard, drive, storage, petabyte, terabyte, server, media, editing

Id: EtZXMj_gUjU

Length: 16min 16sec (976 seconds)

Published: Thu Feb 20 2020

Reddit Comments

[u/XMBomb]

Just curious, did they get the 3D printed bracket from my thingiverse? Would be kinda cool for me to know that I designed something that was useful for LTT.