Enterprise SAS RAID ...on a RASPBERRY PI?!

a lot of people complain about the raspberry pi's microsd storage being slow unreliable and just not enterprise great enough well maybe not that last one but i'm about to show you how i took that complaint and aimed a bazooka at it by going straight to the other extreme that's right literal enterprise grade hardware raid on the raspberry pi compute module 4. i wanted to see if i could get this lsi card working for resilient enterprise grade hardware raid for the first time on a raspberry pi and when i said i went overkill i mean overkill i bought four 7 200 rpm sata hard drives i bought stackable 3.5 inch hard drive cases i bought a giant power supply to give you an idea of just how crazy this is the compute module and i o board weigh just 0.1 kilograms or 0.22 pounds while all these other things weigh a whopping 5 kilograms or 10 pounds and now my desk feels really cramped for space so to anyone who complains again about the raspberry pi having lackluster storage options point them at this video maybe they can also make fun of my attempt before i plug in the sas card i should tell you a little bit about sas and no this isn't the british special air service it stands for serial attached scuzzy but like the british sas it is special if it weren't you'd just see servers with dozens of sata ports on their motherboards and a jungle of sata cables well there are three main reasons servers often use sas though this list isn't exhaustive first these sas storage cards have built-in hardware controlled raid and raid level caches raid is something i'll discuss in depth in another video but these cards often have connections for external batteries for even better protection against data loss due to power failures second these cards have special connectors in this case the internal mini sas or sff 8087 which carries four sata connections in one board connector that leads to a lot less messy cabling when you're dealing with servers with 4 10 or maybe even 20 plus hard drives in one case and third these cards work directly with both sata and sas drives meaning you can use more expensive but also more durable and performant sas hard drives that have faster rotational speeds and better longevity with modern ssds and flash based storage sas is often less appealing from a performance perspective but since many people need lots and lots of storage old-fashioned rusty rotating disc platters are still more cost effective and they're a lot easier to deal with in bulk using sas cards now that we know a little bit more about sas it's time to plug in the card except just like with my four point gigabit network card it just won't fit and no redshirt jeff is not going to cut into the board i actually made him buy me a new pi 4 after he cut mine in half last video and yes i told him you can't make a compute module by cutting the ports off a regular pi 4 and now he finally believes me so i tried plugging the card into the pi via my 1x to 16x adapter the card's cr3 led lit up but when i glanced at the d message logs on the pi it showed the pci link was down that usually means there was some problem thinking it may be a power problem i tried plugging in the card through an external powered gpu riser that didn't work either same problem so i tried plugging it in through an external powered pci express switch and that didn't work either so then i tried the other copy of the card i got i should mention here that both of these cards were server pulls meaning they were yanked out of decommissioned old dell servers and they were given to me with a batch of other cards by jacob hiltz so thank you so much to jacob i'll be testing some other cards that he sent in future videos anyways the other card did almost the exact same thing except at one point all the leds on the front lit up for a bit after a while it settled down and just the cr3 and cr4 leds were lit but on the pi no matter how i powered the card it always resulted in link is down now i should mention i was really excited about getting these cards working i was even considering powering up 16 hard drives and making a monster nas with both lsi cards and i had already run to micro center and bought four drives just to test it out there's a saying don't count your eggs before they're hatched and i think i'm gonna start a new one and that is don't buy your hard drives before you're sassed alas i couldn't get the cards to work at all and though i'm not writing the cards off totally yet i'm not hopeful i'll be able to get them to work with the pie if you want to see all the details of my sas journey and follow any future progress i might make please follow the github issue linked in the description it even has a pretty picture of the card but wait i see you thinking this video still has some time left what else could there be to cover if the sas cards didn't work well here's the thing i had these four giant hard drives and i think i was a little inspired by fezig from princess bride and there there were four white authors and i thought there are four of us if we ever find a lady hello lady i thought i saw micro center shelves and there they were four refurbished hard drives and i thought there are four ports on the i o crest sata card that i'm testing if it ever works hello card so i plugged them in and what do you know after recompiling the kernel the 14th time in as many days i got the card to recognize all four drives and i was off to the races as with everything i'm testing there's a github issue in the pi pci express card database website that details everything i tested with the sata card and it's linked in the description so go check it out anyways i found i had some problems when i went to hook everything together the first issue is cables for the sas card i had some mini sass breakout cables that would have been fine for all four drives but only had two sata cables and i needed two more so i hopped on amazon and bought a three pack it's always nice to have a spare but then i was trying to figure out the best way to power the four drives if i were dealing with a server or a large pc case i might have a large power supply at my disposal but all i had was a measly 4-pin molex power connector and 2 amps i should probably put that on a shirt as my slogan anyways that's not going to power 4 spinning hard drives well so i finally decided to go back to micro center and buy a real power supply finally i didn't want to stack the drives directly on top of each other if i bumped one of them they could fall over or something on the controller board could short out and any kind of sudden movements are risky for old spinning disc hard drives so i also ordered four stackable drive carriers from amazon now it was about this point in the project where i realized i may be going a bit too far i put everything on my desk to start assembling it and that's when it struck me that yes to all of you who are already starting to write your comment out it might be more reasonable to buy a big old pc case get a big old pc motherboard and a big old pc x86 processor and connect things together the old-fashioned way but that's been done before linus tech tips probably already has a hundred different videos showing how to set up boring old raid arrays inside a boring old pc chassis i'm gonna do this inside out so i got to work i made a little room on the table to unpack the stackable drive carriers then i started cramming the drives into the carriers these stackable cases are actually a bit nicer than i expected with rubber grommets and special mounting screws to isolate the drives from the case and keep them nice and quiet and by the way all the things i'm using in this video are linked in the description in case you want to do something as crazy as i am and build a giant pine ass like this one anyways after i got all four drives mounted in the carriers and made a nice mess of my clean white surface i got to work plugging all the sata cables into the drives and i noticed that the 90 degree bends on the two red cables were opposite the bends on the black cables so i decided to use the straight end to plug into the drives with that sorted out i plugged everything into the sata card and realized why most people installing tons of hard drives in a case might like using a sas card instead the mess of individual sata cables that don't quite bend in the way that you want gets really annoying so now that i had all the data cables in place it was time to figure out power i grabbed the giant psu and realized it only had three sata power connectors on its included sata breakout cable luckily i had an adapter cable that split one power connection into four so i was able to plug in all four drives the power cabling ended up looking a little nicer than the sata cables since i could just plug in one cable with multiple connectors and since the wires are able to flex a little easier the last thing i should mention is that with a psu you can't turn the thing on with just the switch on the psu it requires a functional motherboard with a power switch for people sticking the psu and a pc that's no problem but for me i had to rig up a jumper wire between a common connection and the ps on connection in the motherboard power connector i might build myself a nicer jumper or switch for this purpose in the future but for now my little jumper wire here works just fine alright now that i have a much more professional looking rig i booted the pi with my custom kernel i cross compiled the pi kernel and made sure to enable ahci sata support and marvel sata support using menu config for now at least the sata drivers to support cards like the i o crest one i'm using aren't built into the linux kernel the pi ships with i also checked out ubuntu for raspberry pi and the drivers aren't there by default either so for now you have to recompile the kernel to get a sata card working with the new kernel in place i could use lsblk to see all the block devices including sda sdb sdc and sdd all four hard drives right where they should be and what was next well i hate to do this but you're going to have to wait for the next video to see how i used md admin to create different raid arrays benchmark them and set up a raid nest using these four sata drives subscribe and until then i'm jeff gearling there's a github issue and the p and the p and the p compute module four and i'm not holding the card that i'm supposed to show you so let me go get that after recompiling current after recompiling kernel 14th time what somebody's walking above me again oh now one of my kids is screaming the last thing i should mention is that with a pcu a pcu and set up a raid mass using these four sata cards four sata cards

Channel: Jeff Geerling

Views: 112,302

Keywords: raspberry pi, raid, nas, sas, sata, psu, power, speed, disk, hard drive, hdd, ssd, array, mdadm, lsblk, board, computer, storage, lsi, pci, pcie, pci express, network, drive, smb, samba

Id: 1gAUApGaWKk

Length: 10min 50sec (650 seconds)

Published: Fri Nov 13 2020

Reddit Comments

[u/Commodore256]

PCIe on the Pi will kill the entry level 4-bay NASes.