Raspberry Pi Blade crams 64 ARM cores and NVMe in 1U!

i've been running raspberry pi clusters for seven years my first two iterations were a bit of a jungle in terms of wires so my current cluster uses power over ethernet hats to cut down on the clutter and just last month i installed my pi cluster in my network rack but let me show you one of the big problems with my current pie rack even though it only takes one u of rackspace it only holds a few pies and look what's behind it a lot of wasted space the raspberry pi 4 model b is a small hobby board it's not meant to be racked up like a server but what if there was a raspberry pi that used all the depth available in a rack for built-in nvme storage a tpm module built-in poe and more and what if we could fit 16 of these pi servers in the same space as a 5 pi 4 cluster well ivan from uptime lab had that idea and built this the 0.6 beta version of his raspberry pi compute module 4 blade he actually hand made two of these blades and sent them to me after one of his earliest prototypes was damaged in shipping he hand soldered the boards in his workshop and this time they both arrived intact the blade uses a raspberry pi compute module 4 so you get all the performance of a normal pi 4 model b but it's specialized so it can use native nvme storage be rack mounted and increase arm compute density for months i've been waiting to get my hands on one of these boards because for what i do it's one of the most exciting cm4 boards i've seen think of it using blades you could cram 64 arm cpu cores 128 gigs of ram and 16 terabytes of high speed nvme storage in one u of rackspace the same size rack only holds up to six normal raspberry pi 4s the extra depth of the blade allows for some really cool features take for instance this built-in poe daughter card when you're racking pies having to run separate cables for power and network is really annoying so the blade has a built-in silvertel ag5300 poe plus board integrating an off-the-shelf solution for poe means the power circuit is already validated and can supply extra power to the pi and nvme drive more than 20 watts in this case if the board needs it it also has a full size m.2 m key slot so you could put any size nvme ssd into it or even stick in an adapter and put in a wi-fi 6 module or most any pci express device the board even has a tpm 2.0 module built in so i guess it'll run windows 11. but more seriously you can use the tpm module to store cryptography keys verify the boot process and help with encryption it also has a partial gpio header that lets you plug in a real-time clock or rtc module so you can get a persistent clock or even a zimchi 4i for full disk encryption and secure connections to services like amazon's iot gateway towards the rear there's a pwm fan header so a rack enclosure can have speed controlled fans to keep the pi cool the blade also includes more standard ports like a single hdmi port for plugging into display for debugging a single usb 2.0 port for data access a usb c port for flashing emmc compute modules and a microsd card slot for light models without onboard flash rounding out the visible features there's a uart debug header on top making it easy to get console access even without an hdmi display and there are four leds first the standard activity and status leds on pretty much every pi but then there's also an id led controlled by a gpio pin and an nvme activity led i have a samsung 970 evo ssd and a compute module 4 lite with 4 gigs of ram and built-in wi-fi and bluetooth to set things up i'll pop the cm4 on top of the board into the 100 pin board to board connectors then insert and screw down the ssd into the m.2 slot i flashed raspberry pi os 64-bit to this micro sd card and i'll slide that into the micro sd card slot i can actually boot from nvme storage too and i covered how to do that on a similar board earlier this year there's a link to that video in the description in this case i'm okay just booting off the microsd card at least for now all that's left is plugging the board into my poe plus switch and letting it boot up if you're going to use nvme storage with the pi it's best to use poe plus since it can supply more than 12 watts to the pi but if you're not using nvme you might be able to get by with a cheaper poe switch like the small netgear switch i'm currently using for starlink once it's plugged in i can find it on my network and ssh into it logged into the pi i can see all the basic stats ls pci shows the nvme drive on the pci express bus lsusb shows nothing because you actually have to enable the usb port by adding this line to your boot config file on the compute module to activate it i formatted and mounted the ssd and doing a few quick performance tests i was able to read and write data to it at 415 megabytes per second which is as fast as the pi's internal bus allows it to go if you put the same nvme drive in a faster computer it would be able to perform a little better but it's already a huge improvement over microsd or emmc storage and it's even an improvement over nvme drives and even the best usb 3.0 enclosures with a normal pi 4. to be clear this is native nvme storage on the pi it's not using an adapter like the argon m.2 case or other storage adapters built to work with the pi 4 model b this m.2 slot gives you the fastest storage possible on any raspberry pi to use some of the more advanced features like an rtc or the built-in tpm module you have to do a little extra setup work most end users don't know or care much about tpm but it's basically a secure element that can do things like help secure the boot process or store cryptokeys but since the raspberry pi's bootloader is closed source and doesn't support the tpm module out of the box it's a little tricky to use it to its full potential on a pie anyways i was able to enable the module by adding this line to the boot config file and rebooting to verify it was working correctly i cloned infineon's tpm toolbox to the pi compiled it and ran it and it gave me this output so it seems like it's working and it's identified as an slm 9670 but i'm not much of an expert at secure computing on linux so i'll leave it at that for now next up you can control the id led and software because it's attached to the pi's gpio pin 23 running these three commands lets me turn it on and then i can echo a zero to the value to turn it back off using a python script you could manage the led in software to indicate custom statuses or to help identify a particular blade id leds are very common on servers since it's often important to identify a server in a rack where there might be 20 servers that look identical now one of these boards on its own is pretty neat but these blades are built to live clustered in one new rackmount enclosures i spoke with ivan about long-term plans and he already has a mini rack enclosure for it complete with integrated noxua fans but he hasn't settled on a full-size design but he did send me a file for 3d printing a desk stand capable of holding up to four blades i printed this enclosure and put the two version .6 blades ivan sent plus that early beta board that broke during shipping and even this the original prototype board design it's a lot different than the version that's getting close to production it's always fascinating to see the progression from idea to prototype to production and i love the personality ivan and jackson to the designs seeing the vulcan salute the delorean from back to the future and even grogu i'm guessing the final design won't have a copyright infringing silkscreen though so this board is awesome but here's the bad news these are two of only a few blades in existence you can't get them yet right now ivan is working out a few more bugs so he can build a final production version and his plan is to start a kickstarter campaign soon since that campaign isn't up i'm going to add a link in the description to his email list so you can be notified if you're interested once the campaign is live i'll be sure to update the link i can't wait to get my hands on a few more boards and rebuild my pi cluster for a new 2022 edition that crams tons of arm compute power into a 1u rackspace with tons of high speed ssd storage follow uptime lab on instagram to find out more and until next time i'm jeff gearling 64 armed cpu chords chords not chords most end users most end users i don't know why it's so hard to say i'm not saying that um anyway too many p's the kids are only gonna be gone for a few more minutes gotta wrap this up

Channel: Jeff Geerling

Views: 475,375

Rating: 4.9609475 out of 5

Keywords: raspberry pi, arm, arm64, server, cluster, kubernetes, k8s, k3s, render, farm, nvme, ssd, tpm, tpm 2.0, windows 11, 64-bit, os, oss, uptime lab, uplab, pro, blade, slot, enterprise, 1u, rackspace, rack mount, rack, 3d print, custom, poe, poe+, zymkey 4i, amazon, aws, iot, internet of things, silvertel, solder, make, maker, build, noctua, enclosure, design, sbc, computer, compute module, cm4, compute module 4

Id: zH9GwYZu_aE

Length: 8min 38sec (518 seconds)

Published: Wed Aug 04 2021

Reddit Comments

[u/ds679]

Can you (or someone) share what use cases this would be good for (instead of a standard/typical rack mount server)?