The ULTIMATE Raspberry Pi 5 NAS

I've built a bunch of Raspberry Pi nases from a little tiny all SSD Nas to the biggest one on earth the petabyte Pi Project but the pi4 and compute module 4 were just barely adequate I could never get even 100 megabytes per second over the network even with ssds the two most promising projects the wire truste SATA board and radas taco were both dead in the water they launched right before the great P shortages when you couldn't get a Raspberry Pi For Love or Money but the Raspberry Pi 5 is here now it's faster it has PCI Express and best of all you can actually get one yeah it's a little more expensive than the pi4 but with off-the-shelf 4bay nases costing 300 bucks and up could we actually build a pine nass for less and would it be any good well today I'm going to see and to do it I'll use this tiny SATA hat that RADS Us in this costs 45 bucks and it's already shipping add a 12volt power supply a Raspberry Pi 5 a fan and micro SD card and we have a tiny Nas for less than 150 bucks but will bottlenecks kill this thing like they did with the pi 4 I mean the pi five only gets a gigabit those other nasas can do 2.5 and they have hot swap drive base and vendor support so yeah comparing just on price alone is silly there's always going to be tradeoffs when you go DIY but this thing should have a lot fewer compromises than the jankier builds I did in the past at least I hope and 2.5 gig networking I might have a fix for that I'm going to put this thing together and see if it could be the Ultimate Raspberry Pi 5 Nas I do not know exactly what tools will be required and I don't know what's in the box hopefully it includes everything I need uh but uh rans usually does a pretty good job including all the little bits and Bobs you need for this looks like it includes this extra cable this is after all the penta SATA hat so five SATA connections I have four drives here uh but you can add on another one using this strange external I guess this might be eeta or something uh but it has SATA and power from this port something important to uh think about is how you're going to supply power to it uh I know some people in comments have mentioned oh you need to supply power to the pi and this board but no uh I believe that you can just power this board through the 12volt barrel Jack or through an ATX MX power supply here so if you have have it in a PC case or something you could do it that way and this will power to the pi5 through the GPI opens uh this should be able to provide adequate power as long as the power circuitry on here is good enough to take that 12volt signal and give a clean um you know 3 to 5 amps on the Pi's 5vt rail now this doesn't have the normal PCI Express connector that you see on the pi5 so the pi5 has this little this little guy here uh this has a much larger connector with more pins that could be an interesting thing uh I believe that they have an adapter for it though so yeah here it is so this is called an FFC or flat flat flexible circuit board and it looks like they've included two which is nice because these uh these these little connectors are a little bit delicate you can see how thin they are they're kind of you know they're kind of like paper thin uh but these are flat flexible circuit boards or ffc's and they connect from the Pi's PCI Express connector here over to this guy here and the GPO pins over here are going to provide power to the pi at least that's my hope there is a getting started guide on here but uh I you know I'm going to YOLO this thing and see what happens one important thing whenever you're doing these is make sure you get the connector seated all the way and it should go in pretty easy if you're pushing hard then you're going to break the cable so don't do that if you're pushing hard you might need to pull this little connection up and always do it on both sides so that it doesn't come off because if it comes off it might break and then you would not have a way to hold the cable down you push down on this little top part and uh this cable is now a fixed to the pi very well and then I'm going to plug it into here so it looks like it goes like this the funny thing is these these kind of connectors are often used inside of of you know cameras and and other things that are put together at factories and uh they they're very careful they have their methodologies they even have tools to help with it uh when you give these things to people in in the general public like you and me we tend to break our first one so I guess it is a really good idea that they included a second one here uh they probably have some screws too let's check yeah there's little Kit full of screws here there's uh some standoffs and things and then now I'm going to put this in I'm going to carefully put this over and plug in the GPO pins that provide Power uh but that fits nicely together uh there is a connector here for an OLED and fan control board that sits on top of the hard drives at the top they don't have that available yet I I think they used to make it I don't know um if they needed to revise it for this or what but uh I asked about it and it's not yet available so it' be nice to have that especially these are not that hot of drives but if you use hard drives if you use 2.5 in hard drives and those can get pretty toasty and it's nice to have a fan blowing air over them um I just realized I don't have any fan on the pie itself and I probably should do that because it could get pretty hot and toasty inside here let's get our little active cooler here I hope this will fit I I don't know if there was a warning against using this but uh the pie does need some sort of cooling whether it's a heat sink or a fan and there's uh there's no fan built into this it' be cool if there was a little fan under here or an option for one but it doesn't seem like that's the case okay please still fit looks like it will fit oh no you know what the uh the barrel plug is just just touching on the top of the top of the heat sink there's there's literally uh it's just three three of the fins on the heat sink you know what I might do might see if I can bend those off take this back off again I'm going to pull this connection off this is a terrible idea I would not recommend doing it just bending this back and forth there's one shouldn't affect the performance that badly I removed the middle portion from from the Middle Point up of these three little fins on the heat sink and uh there's a side view of it you can kind of make it out it's kind of hard to make out sorry about that but let's get this all back together now and see if it fits okay this time if I go down it can go down all the way and look at that that's just enough clearance as long as it works in the end it's all good I use this huge huge guy just give these a little snug generally I'd use a nut driver for this but this works in a pinch literally my topown recorder decided to corrupt the rest of the video so I lost all that footage but in that footage I mentioned the board uses the jmbb 585 pcie gen 3x2 controller which means even if we upate the pi 5's bus to gen3 from its normal Gen 2 will miss out on a little bandwidth and also the kit comes with two side supports that hold all the 2.5 in Drives together though there may be a case available at some point in the future they actually had one in the past when it was sold for the rock four or the pi4 I think but I'm guessing that they'll have to make another batch if they get enough interest in this new version of the pent SATA hat okay so everything is put together now it's all looking nice and I I think there will be enough air flow uh there's holes in the sides holes in the middle so enough air will conve through for these drives at least and I have a 5 amp 12 volt power supply this should be adequate for these drives and the Raspberry Pi 5 I'd budget maybe 3 to 5 watts per drive or if you have 3.5 in drives maybe a little more and you might want to get an 8 Amp or maybe even 10 or 12 amp power supply uh but definitely don't use like a 2 amp power supply and expect this to work it's going to have all kinds of issues I also have raspberry pios uh 64-bit light version and I might try open media Vault I'm going to take the micro SD card and put it into the slot and then I'll grab this power adapter now one another reason why I'm over at the desk is I have my little this is a zigg third reality zigg outlet that has power measurement built in which is very handy for testing I'll go ahead and bring that up on here if I go to home assistant and then go to Power you can see that right now there's zero Watts because there's nothing plugged into it power is going to come in looks like they they wanted to align the power with the USBC Port not that that matters uh first I'm going to plug in network and I'll plug in power and we'll see what happens hopefully no sparks all right I have a green light on the board and the pi is booting up power usage is up to 14.2 Watts at boot and now the pi is doing its reboot so it's going to reboot a couple times this first time that I I turn it on uh because it expands the file system to fill up the S the micro SD card all that kind of stuff so we'll fast forward a bit until it's all booted up and then we can log into it on the network and see if it's actually working I don't see any lights there's just one green LED on the board over here but I don't see any other lights so I don't know if there's lights per hard drive so I'm going to log into it and we'll see what we can see sshp at pi- n.l there it is and if I say lsblk hope hopefully we see those hard drives no we're not seeing them let's try lspci and I'm not seeing the device at all I don't see any errors in here let's go to uh the the URL on this box and see if there's any other tips that we're missing rock. sh Penta hat pent hat so we did that we did that oh so maybe I should actually do that let's try that go in here you'd think it would do it automatically but it does not so we're going to enable PCI Express save and reboot so save that and reboot so let's check again there we go we have 1 2 3 four hard drives and if I Say LSP I can see the J Micron SATA controller now uh right now it should be PCI Express Gen 2 we can check that with uh pseudo lspci - vvvv this is going to give us all the information about PCI Express devices and if I go up to here this is ahci that's the kernel module for the uh SATA controller and we can go up to the top section see it's j Micron jmbb 585 and if I go down to link capabilities it says Speed 8 gig transfers per second width by two that's pcie gen 3x two but the status says it's 5 Giga transfers by one so definitely less bandwidth than the chip is capable of so I'm going to try PCI gen 3 and I can do that following my own guide if I go down here turn that on like this and reboot and we'll see if it gives us gen 3 speeds instead of and two speeds which would give us the maximum performance that we can get on the pi5 I have four drives that have nothing on them uh I'm going to try we should probably just Benchmark the drives first in like RAID 10 just to see what the maximum speed is or maybe even raid zero so let's let's do that that'll take a couple minutes and uh we have blinking so you can see that the the LEDs actually do work I didn't see those when I was looking earlier but it has some LEDs and you can see them blinking when the dri or access so nice job uh I should check it does feel a little bit hot uh in infray I found them at uh CES and they actually sent me home with a couple goodies uh this is the P2 and it's uh the reason why I wanted them to send me home with one to test was uh it has this Snap-on macro lens that you can see individual resistors or things on your PCB very closeup which is kind of cool uh but their software is a little bit little bit bit iffy um not the best software that I've used for IR cameras uh but the camera itself is really good quality and works better than my old seek thermal but let's check the temperatures on here and it looks like the drives themselves well they're a little bit reflective so we might not be seeing the actual Drive value but the board the board is up to 50° or so the uh sta controller is down there it looks like it's the hot part of this thing and it is getting up there to 60° C so it might be good to have at least an active fan blowing down on top there's the cold soda can 16° C and there's the hot SATA chip so I'm going to put this cover on and see up nice and close if I get in there yeah we can see that the chip itself is 60° C so it's pretty toasty in there yeah I would definitely do a fan or heat sink on this if you're going to deploy this long term another fun thing with the thermal imaging is you can see all kinds of fun details like you can see that this is where my my hand was resting and if I just put my hand on the table and take it off there's a handprint and apparently this little this little screen on here also generates a teeny tiny bit of heat and now it has my fingerprint on it which is also warm looks like the formatting is finished and what's our next step Here Mount the array okay Mount raid zero so now let's do a disc Benchmark on it and we'll run the dis benchmarks and see how fast this array can go okay here goes fio hey that's not bad at all 87 850 to 60 megabytes per second and that's megabytes so let's see how fast it was in megabytes uh almost 900 megabytes per second across all four drives in rate zero of course uh but uh random reads of 687 megabytes per second and random rights of 758 and then we have 4K block size uh 44 Megs read and 152 Megs write at 4K which is not bad at all um I'm interested in seeing I think what I'll do is I'll just put an a samb share on this and we'll see if we can saturate a 1 gig connection continuously restart Samba and create a password now I should be able to connect on my Mac py. local uh we'll do the shared directory here it is so I'm going to copy over a folder with all of the footage of the build that's 100 gigs and let's check this out let's see how fast it is that is line Speed 110 megabytes per second is pretty typical let's see if it can keep keep up that data rate I can smell that like slight off gassing here so I I do think that I would put some sort of cooling on here just for that jmbb 585 chip on my other NASA over 1 GB you can just hammer it and it'll stay 110 15 megabytes the entire time this is a lot faster than the pi4 nasas I've set up before though and we'll just let the screen recorder keep going at 18 minutes and we'll just keep moving while that's copying I want to take a brief opportunity to tell you about Open sauce Open sauce is going to be June 15 to 16 in San Francisco and I'll be there I'll be there along with a ton of other creators in the maker areas Electronics hacking all kinds of fun things if you want to go there's a way that you can get in for free and you can come to the party that's beforehand where all the other YouTubers and everyone will be present uh if you want to do that you can apply to be an exhibitor they have tons of space for exhibits this year it'd be really cool to see your project so if you want to do that go to opens sauce.com and apply to be an exhibitor otherwise you can also come as just a normal person who's not exhibiting things too so hopefully I'll see you there June 15 to 16 if not I will definitely be posting some things on Twitter and maybe something on YouTube I don't know so make sure you're subscribed it copied everything over to the pi now let's check the read speed I'm going to copy it back into a different folder on my local computer and we'll we'll see if it can give me 110 megabytes per second oh look at that it's giv me 122 which is a little faster than the right speed and you can see that the drives are reading uh pretty much flat out right now I don't know if that'll fill up the cache but you can see that the the data is Flowing a lot more smoothly coming off the P than riding to it so there are some bottlenecks I don't think it's Samba and I don't think it's the drives themselves I think there's a bottleneck somewhere in the Pi's kernel or something when it's riding through because I had that problem on the pi4 but on the pi4 it wouldn't even hit like 120 megabytes per second all the time uh but reading that's not an issue at all here we're we're cranking at 120 megabytes per second I deleted everything off of there and uh it looks like obviously read speeds are much more consistent than right speeds uh but I'm going to try something else that uh I mentioned at the beginning of this video what about 2.5 gig networking now Pineberry Pi it makes the hatn net 2.5g this is a 2.5 GB hat for the Raspberry Pi 5 uh but you'll probably already notice there's a problem it has one PCI Express input there's only one PCI Express connector on the Raspberry Pi 5 how do we solve this problem because this needs that and I want to put it on here too to see if I can get 2.5 gig networking well I can try the Hat brick Commander from Pineberry pi and yes they sent me these things I would be buying them myself anyway but I'm going to disclose that radza sent me this and Pineberry piie sent me this I'm testing these things out to see if they can work together and do some crazy things uh but Pineberry also sent me all of these extra cables of varying lengths one thing that can be a problem with when you start connecting multiple things together is the PCI Express signaling so I'm going to try to use the shortest cables I can for these experiments but I'm going to basically put this which is a PCX Express Gen 2 switch off of the Pi's bus and then connect one connector to the SATA drives and the other connector to the the Hat net 2.5g the downside is this is going to make everything be PCI Express Gen 2 speed inste instead of three so I wouldn't be able to get 800 megabytes per second on these hard drives but uh on the flip side this is 2.5 gig networking and if we say let's say 2 gigs for networking and 2 gbits for the hard drives we might be able to do that to almost saturate 2.5 gig network if the pi5 can support that I don't know if it can or not I don't think it will be able to but we'll see uh if any of this even works it might also not have enough power I don't know uh but I'm going to unplug this okay we got that connector out of here there is some risk here if we are mixing these cables from different vendors and connections there's a little risk that Uh something's going to go wrong but hopefully that doesn't happen it's it's definitely not my finest work there's an LED on here and I see a light on the switch and there's a power LED on the hat brick commander and there's lights on here let's see if this is actually going to work lspci hey look at that so we have the switch here we have the SATA controller here and we have the 2.5 gig controller here let's do uh IPA and we have an IP address on that so let's do a I perf test now we're getting 2 GBS it's not 2.5 gbits but it's not nothing so coming back only 1.6 gbits that's not horrible it's still more than a gab this is probably going to get 2.5 GB if you connect it straight to the pi I think that some of the overhead comes out of that packet switching uh that is running to the drives as well so if I say lsblk we still have the drives and they're mounted so we'll see if we get any faster right speeds it's doing 110 117 that's about the same as what we were seeing before so we're not getting faster than a gigabit over the 2.5 gig connection at least for rights I do see a few Peaks up to about 125 megabytes per second so better than a gigabit and it it's interesting the uh the overall rate seems a little steadier with the 2.5 gig I maybe the Pi's internal controller is a little funky but I don't know um but it is giving it's giving us a little bit more on the right speeds I'm really interested to see the read speeds though hopefully we can get more than one gabit let's check there we go 217 megabytes 250 megabytes per second that's more what I'm uh what I'm expecting out of a a 2.5 gig connection so this can put through that data it's it's interesting I think it's pulling from Ram because I don't see the drives blinking at all here uh it's probably copying all this data from RAM and now it's hitting the drives and you can see it dips a tiny bit there so down to 230 megabytes per second so Linux usually caches files in the ram as it's copying them back and forth so that if you have a file that you're accessing a lot it's a lot faster uh but now that it's hitting the drives it only dipped down 10 megabytes per second so that's not bad at all so for a read heavy nass this this isn't looking like that bad of a setup now that I know that uh everything is going to work on here Hardware wise I think it's time to put omv on here and see how that runs I haven't used omv 7 yet so this will be new for me I don't think it's that much different than omv 5 and six but uh let me grab this script and go over here and this hopefully will just work I'm going SSH into the pi and just paste in their script the installer and here it goes let's check power consumption so during the install it was using between 8 to 10 watts and it looks like the Baseline for this build is 8 watts with the 2.5 gig network adapter and everything else uh but let's go to past. local and does this work maybe I have to use the IP address let's try that well there it is I guess it was still booting up okay so that was not the problem there so admin and open media Vault are the password logging in there it is there's no dashboard that's okay storage is where we should see our diss they should show up yep 1 2 3 4 all of them are 8 terabytes and uh I want to create an array file systems is this where we create create and mount a file system ext4 but I want to create a raid array how do I create a raate array am I totally missing something I thought there was a thing over here for creating raid but I don't see it anymore uh what does this say See this has raid management but I'm not seeing raid management anywhere do you see raid management anywhere we could try ZFS instead of raid but that's instead of like MD admin raid so we can try it out on open Medi Vault I've never tried it on on omv before uh but we'll see how it works here I like this little uh end of line here I guess a nod back to Tron the 1974 version and we'll do raid Z1 since we have three drives uh a raid Z1 will use one drive the equivalent of that for parody data that way I could lose one of these four drives and all the data would be intact but here we go it says pending changes 21 terabytes available let's apply this so now the tank should exist compression is on I don't know if I would need compression on but I'm not going to mess with any of that right now uh if we go to pools is there anything else I can do tools what do we got so you can scrub it I don't know if it automatically scrubs in here but it gives us the uh pool information that's nice so this is a it's a good interface it's it's not the maybe not the best thing ever and it I don't know if it comes with schedules and Things by default but it'd be nice to have a scheduled snapshot and uh pool scrubbing scheduled that might be something that you can figure under scheduled tasks yeah so you'd have to you'd have to do some of these things you'd have to add your own scheduled tab tasks it' be cool if that added some things by default but I I can see why they don't as well uh but now let's add a file system so we have one tank ZFS I'll add uh shared under tank shared and we'll just set everyone read right right now save turn on SBA enabled 10. 0.221 okay so it wants me to use the IP and there's our shared volume so let's uh I'm going to copy some stuff over to it I have uh this this folder has 100 gigabytes so I'll do that and here it goes so it seems similar to the uh the copies that we were getting with raid zero uh it is it it's interesting it it it goes a little bit faster sometimes than those copies were so I'm wondering if ZF is caching is actually helping here so far I'm pretty impressed um I think read speeds are where this wins uh right speeds are where this loses a little bit because you're not going to be able to get full 2.5 gigb networking on that but uh but it's better than I was expecting and the big win for me besides the fact that this can be made smaller if we kind of reconfigure these boards uh the big win is the power efficiency because right now uh we're using 15 or 16 Watts typically the other nasas that I've built uh using you know pre-built nasas they use they use 10 to 20 watts idle and they use 25 to 30 Watts when they're doing a lot of stuff so this little guy is only using 16 Watts doing the same amount of work uh which is probably about half of what most uh pre-built nases would use on the flip side if you build a Nas with the RK 3588 chip you could probably get even more efficient and more speed so there's a couple boards out there that are interesting uh that I might take a look at at some point but uh the nice thing is all of this this is all really well supported like the software just clicks some buttons and you have everything working I haven't always had that same uh kind of experience when I'm using the rock chip boards uh some of them are getting pretty good though I'm going to go ahead and let this right finish and then I'm going to do a full read of that 100 gigs of data and uh we'll see where we end up at the end of the uh copy it looks like the the system used 22 Watts for a little while while it was doing some sort of processing I don't know what ZFS was doing there maybe maybe that was part of the compression I don't know it's a lot of power to use at the end there uh the actual performance was interesting after that initial part uh where it was faster than raid zero it actually slowed down to a tiny bit slower than raid zero over that long rest of the copy and that's why it's good to use a large large file to test the actual performance of your system because especially with CFS it's going to cash a lot in the beginning in Ram and that throws off how fast your actual disc array is uh but the CPU usage was not too bad uh power consumption was down around you know 8 to 16 Watts throughout the whole copy uh but in the end the file copy uh was 74 megabytes per second with CFS in raid Z1 and it was almost 100 megabytes per second in raid zero now that's for the writing uh which is is going to be a little bit slower with a setup like this uh read speeds for both are practically the same it's just basically line speed it's it's not hard at all to do the reads so this is a little embarrassing all those conclusions I have are based on the fact I was benchmarking this all on a Mac and uh I switched to my Windows PC and I was able to get almost line speed for 1 gbit uh file copies writing to the pi and 150 megabytes per second writing over the 2.5 gig Network so that's uh that changes my perspective a little bit on it and I think the biggest takeaway is don't use a Mac for benchmarking Network file copies even if it has 10 GB networking and everything else on it seems to be fine uh Mac OS for some reason is not great with file copies and I have a whole blog post and I have more details in the GitHub issue linked below but let's get back to the video it's not inconceivable to build a system like this Allin this one is still under 200 bucks total including all these extra boards and things so uh but but it always goes back to DIY means you're responsible for the software you're responsible for maintenance and updates and all that kind of stuff anyway uh that was a fun experiment and I plan on doing some other fun experiments now that I have this little this little board here that lets me uh split up PCI Express Lanes and uh we'll see how we can bend the pi 5's PCI Express Bus it would be really cool to see a compute module 5 expose even more but we'll see what happens whenever that comes out um I know that that was a big change from the pi4 to the compute module for it gave us PCI Express now we have it on the pi5 but I think we might be able to do more in a compute module form factor but we'll see until next time I'm Jeff Garling

Channel: Jeff Geerling

Views: 1,328,257

Keywords: raspberry pi, zfs, nas, storage, homelab, cheap, inexpensive, ultimate, network, attached, networking, share, shared, drive, ssd, nvme, sata, hard drive, penta, radxa, taco, waveshare, board, compute module, pi 5, pi 4, cm4, cm5, speed, performance, omv, open media vault, filesystem, sharing, smb, samba, nfs, setup, guide, test, how to, tutorial, testing, plugin, plugins, installation

Id: l30sADfDiM8

Length: 32min 14sec (1934 seconds)

Published: Thu Apr 04 2024