Deploying ANOTHER PETABYTE of Storage!

as promised I have brought you guys the one and only petabyte project - and the one and only Anthony Anthony our task today is very simple we have two hours to have this petabyte project deployed so that the editors and the shooters can offload a bunch of footage off of the existing vault onto this one so we can load balanced things a little bit easy right easy [Music] so uh go wave the wand what what do you want to do what you do well I don't know how I can plug it I mean it's not it's not powered I can't like just cast a spell and we're gonna do something really similar to what we've got for existing petabyte project where we're gonna do 4:15 Drive the devs in raid-z too so so that you guys know that allows us to lose up to eight drives total before we lose any data and that has come in handy yeah and the resilvered was a bit of a nail-biting process let me tell you that took many weeks I was thinking of adding an octane PCI Express card as a slog so that would be a right cache effectively that doesn't depend on DRM so that in the event of a sudden power loss we would still retain all the data I don't know how big we feed that though 480 gigs that's the smallest obtain drive we have okay okay I'll go grab don't forget to sign it out Hey got him oh they break oh yeah for a server chassis that's like hundreds of dollars this is not an amazing experience so we're ready to power it up sure it's actually not even crazy loud no I was expecting much louder did they send the credentials for login well let's try root and password okay we should probably change them today boys first of all you shouldn't be logged in as root well I mean just for like initial configuration it's the matter yeah do RM dash RF slash don't do that oh wait you were gonna ruin my stuff now the original plan was for 45 drives to build me a custom 75 drive unit right but Seagate did send 75 drives so I was kind of thinking you know what would be cool is if we took an old decommissioned astoria nater and actually created a new cluster with this one and that one I'd have to upgrade it obviously so we go epic with the other one as well and what do you think No yes good bad they said not to do it over a petabyte let's do it hold on a second we might need drivers for that Mellanox one what are those ten gigabit cars that are in there they're just the hormones I could try another slot okay you want to throw the power switch to make sure it doesn't wake back up yep and now okay that's not a power switch I asked you if there's a power switch well there is a power switch it's not hey how's it going I found another 60 drives tornador so now we could have matching ones are they really matching well if I change out all the hardware in this one they would be somewhat matching yes somewhat yes the older model though I was kind of thinking while you're working on this I refurb this I don't think this should take very long all we have to do right now is last words let me see if I've got the Mellanox card now yeah there we go oh nice so just moving it to the other slot worked yeah I hope that just rebooted on its own probably fine famous last words yeah thanks for that oh man do you remember when 45 drives you see is non-ferrous screws me too this is super annoying I'm gonna replace every screw in this case when I rebuild this okay well I said this probably wasn't gonna take very long I didn't realize that we had to download so many updates so here we are Oh failure wait God why meanwhile I'm extracting this motherboard oh shoot I didn't unplug the front panel connectors or the 24 pin we are totally not gonna finish this today Oh package G deploy because it's not available in the mirror anymore what's G deploy I have no idea what the crap oh boy that is a dusty server dusty boy ah ah go away just saying to use bluster ansible so I'll just use that hey there we go feel like I'm playing operation don't touch the sides okay all lit up yeah the driver showed up Wow yeah it was just a different command sweet thanks Anthony go nuts that was our motto at Best Buy was have fun while being the best how'd that work out for you Weaver the worst-performing store in Canada yeah like all the contacts are still there if you could get enough mounting pressure on the pins he should be able to work what if we jammed something in there that might be something non-conductive they have presses against there I found an old junk 3d print there's actually bit on this side too whatever hit that thing hit it pretty hard so he just ripped the memory out of it without pulling the tabs first no I think it was during transport at some point well they're physically in and see what happens show me the 128 gigs defeating intensifies uh oh yeah we got it we repaired the memories love repaired it's such a dumb fix but is this is totally gonna work thankfully hot glue is non-conductive and it melted the filament a little so didn't think of that hot glue it's it's hot filament it melts our hot glue fix is applied so this should boot up with 64 gigs of RAM right now could got it the scale of the project has changed a little bit so we're gonna cheat ever so slightly and pull three of our eight terabyte SSDs and put them in this second server we actually pulled out all of the eight gig sticks of RAM that were in the servers and we are replacing them with 32 gig sticks so each of them is gonna have 256 gigs around yeah hey you only live once and I think that's pretty much all that I've changed other than that the last time you guys saw this it had old hardware in it and now it is running brand-spanking-new 16 core epic hardware and I've got another 256 gigs around that I can throw in this puppy okay well we got 256 gigs around okay sweet this is working then theoretically there's nothing wrong with what I'm doing here but I am officially the jank King this power supply has only two SATA connectors and I need to plug in five drives so get this I'm going SATA mail to say to female 2 molex mail to molex female to quadruple SATA over here battle power the for regular SATA drives and then I'm using the other one for the obtained row it's kind of jam that in there yeah I mean thankfully SSDs don't take that much power exactly that's what I'm talking about who's gonna say anything all of our viewers yeah I am just about done building this puppy I've got my SATA cables run just casually doing some cable management with my LT t store comm cable ties fans are clear and they're spinning everything's good to go all we got to do is load it up with some drives oh I guess I need to find all the drive mounting hardware I put away a panel that has some screws that go with it I put a little magnet on there with all the screws stuck to it see that free check to Dave three of this epic shoot what do you think is this just gonna boot up I would say 95% yes I mean we haven't even checked the RAM compatibility yet okay well that's a bad sign go ahead cut let's go ahead and cut go ahead and cut no dice on getting it to boot so I'm gonna start by ripping some extraneous hardware out and just see if at least we know the CPU and RAM work you know I don't think that's doing any better but according to the manual I shouldn't need any of that stuff thread cut you want to get a closer ah my foot okay all right we're back where we started cut now to just do one HBA at a time one cut what is this g3 do you know cutting cut go ahead and cut Oh weird just cut this is very boring troubleshooting after a BIOS update and ASIMO's clear its up everything is detected what you think I'm just gonna friction mount it no no I'm gonna put some hot glue along the top of the card edges and I'm gonna stick it down on it don't worry about it it's all good I read about it in a textbook somewhere these babies ain't gonna deliver themselves III like literally cannot tell if you're joking people claim they don't love the gore but they love the gore it's in it's done we could put rides in it now right yeah I mean theoretically it's all plug right heck yeah oh right okay so no we should just do this in the server and that one's a lot harder to remove the drives from and we have to remove the drives in order to transport it so I think that's the extent of the testing we can do here what about this we're gonna have to remove all the drives and then put it back together in there okay do we have to put them back in the same order yeah I think so okay unfortunately we're out of power Bloods the good news is I did buy a second one that isn't mounted so we can put that in but the reason I didn't bother doing it was that it was gonna be a royal pain in the butt checklist put in new PDU for more power move Albion server up figure out all the KVM cables and labels so that everything corresponds again and then add two more servers easy right easy logic when you put it like that yeah oh wait no missed up a bit good in I thought worse man am I gonna have to just kind of jam it in there I'm glad you guys can't see this right now okay we've got power we've got KVM so we can throw them in plug in network and then we're gonna go alright this is awkward mind labeling that well I put the rails in pull out just a little there we go easy-peasy boom box 2 wins all I wanna do is sit back and kick it all the cops wanna do is write me a ticket wait this is for why did I put it here we got to move it up like blood and cut and now to make transporting this a little bit easier we're going to take out all the drives put them on this little card here and transport this separately we're gonna make the biggest cluster cluster we can by filling it up with some of the drives that we're just gonna return later I guess yeah I'll just grab it sure yeah you know what I got it don't worry about it no problem okay okay yeah boom box don't big camper you yeah that's fine I'll just do this that's fine I got it oh I thought you were being no now the mean trick is to put these all in the right spots all the storage will ever need ever wow that's a total capacity of like 300 drives I noticed that this one's flashing green though Brendon cut let's see so you change the Mellanox card oh yeah so we got the static IP 44 and we are able to ping rock on now we just think the same thing to Delta 3 how long does it take though however long it takes for me to manually oh my god you have to key those in yeah so we're picking this up tomorrow yeah okay yeah okay works so close the good news is that Anthony found a script that makes it easy to import all the drives and it will say nothing here buds yeah no way they are the most eastern Canadian tech company of all time wait so how do we have hold on a second but that doesn't make any sense because that's whatever that's happen ever that's doing I have no idea its lifeline time yo Bret long story short we tried to use the new script okay awesome so it's an older chassis and I repurposed it with new hardware so there's a drive okay give me a second I'm gonna do some open-heart surgery here moving all the cables is gonna be a little dumb don't fix yourself all right having fun yet 45 drives never fix the software a problem with hardware also 45 drives can you unplug literally all the cables and put them in the other way in their defense this is not a supported configuration so are they just in reverse order on this motherboard because I think Supermicro numbers them from bottom to top I think so you're coming up for LTX right yeah hell yeah okay cool I'll see you then then okay sweet they're all showing up so it was just a matter of it was just a matter of receiving all the cables in a different order sort of yeah sure yes so we're ready to pull them together and then start Samba and then we can see over one petabyte accessible over the network we're ready to pull them together they recommend that a single break shouldn't be more than a hundred terabytes in size wait so this is good probes are good probes are very good okay we just probed our peer probe success Rob success let me do the other way around peer probing do you want to see it I do want to see it now Oh last time we had a petabyte of raw storage but because of overhead lost to redundancy and whatever the case may be we ended up with what like which 600 some things yeah terabytes this time 1.2 petabytes of storage it's ready would you say that's a new PB it's a PB personal best yeah definitely so I'm just gonna take a quick test file here how about this three gig file we going so yeah it's not quite ten gig speeds at around seven hundred and thirty seven hundred and fifty gigabytes a second but considering that we're writing to magnetic storage I am pretty pleased oh we never installed pulse wait on the server and you just get Anthony installing pulse way it's up already just like that Delta 3 21 hours of uptime don't you wish the cluster setup was that easy I know right so thanks for watching guys if you enjoyed this movie check out our video rolling out our high-speed nvme server it was almost as headache-inducing as this one actually more so this this was relatively painless most of the problems were caused by me deciding that we were gonna roll two servers at once instead of just one so yeah we're gonna have that video link below thanks Andy he's been a trooper he's been a trooper for this

Channel: Linus Tech Tips

Views: 2,257,691

Rating: 4.9506168 out of 5

Keywords: storage, server, petabyte, hard drives, spinning, 4U, rack, rackmount, 45drives, storinator, 60xl, seagate, ironwolf pro, exos, array, zfs, slog, zil, l2arc, arc, cache, SSD, optane, deployment, configuration, glusterfs, gluster

Id: fbipJUJLzpE

Length: 15min 55sec (955 seconds)

Published: Sun Feb 23 2020

Reddit Comments

[u/ConceptHut]

Molex to SATA. Oof