All-flash NAS fight: DIY or Buy – Round III!

a year ago I put up a Raspberry Pi against my Asus dornest but the pie just couldn't keep up over the network I got under a hundred megabytes per second but this year there's a new single board computer in town the rock 5 model B this thing is way faster internally it can write data over a gigabyte per second could I build a better DIY nest with this thing maybe my original plan was to upgrade my old nest with this new Dual Purpose 10 gig m.2 card and put it up against The Rock 5. the card is pretty neat and it made it so I could have both 10 gig networking and two nvme cache drives on my six Bay Nas but well two things changed my mind first case store came out with this thing and it's even faster it's got all nvme flash storage and this model has 10 gig networking built right in second a maker named Rick Johnson sent me this thing the pocketness it also has m.2 flash storage but with a few compromises I'll get to that later to be clear here Asus store and Rick both sent these to me for testing there was no other compensation and they had no input into the content of this video what's interesting about both of these Solutions is they only use SSD storage good ssds at least have great Random Access performance they use less power than hard drives and they take up a lot less space they are more expensive per terabyte but if you can stomach the price or you just want a quiet all flash nest for video editing or a small media library they look great on paper first up the pocketness now this is a very early prototype Rick is working on a new design that fixes the problems that I'll point out but this thing is a six Drive Satan s it uses these tiny m.2 SATA ssds I assembled this origami of pcbs and then I installed six of these team group one terabyte drives these things aren't that expensive they only cost about 40 bucks each all in this whole kit was six terabytes of storage cost me 600 bucks but this thing doesn't come with a case at least not the Prototype I have and it's come completely DIY meaning software and support is on you I booted up a copy of Army and Linux then I installed open media Vault from scratch open medivault is pretty good but it's a bit less intuitive than the software you get with a pre-builtness creating a storage array with all those m.2 drives requires a bunch of clicking around and one thing I don't like at all about omv is how almost every action you take it takes a few seconds then this warning pops up saying you have to apply the change then that takes a few seconds and finally you can be on your way again I wrote up an entire blog post on how to set up omv on the rock 5 model B so go read that I I won't cover it in detail here omv's interface quirks were a minor issue but what wasn't a minor issue was how half the ssds here had issues that's because this complex origami assembly resulted in a few broken connections from Shipping that's a big reason Rick is redesigning the pocketness these two drives on top don't get power and one of the slots on the second board is just flaky meaning omv would just see the drive drop out sometimes I decided to just use these these three working drives and raid five that means I have two drives of storage space and a third drive for redundancy I could lose any of the three working ssds and not lose any data on the Raspberry Pi raid 5 was pretty slow that's because every time you write data the CPU has to do a data parity calculation then write that extra bit of data to the third Drive luckily this rock 5 has a faster CPU and it's not the bottleneck anymore using the built-in 2.5 gigabit Ethernet I could write up to 1.2 gigabits over the network though the average was more like a gigabit just over 100 megabytes per second so that's a win over last year's Raspberry Pi but still not as good as my old Spinning Disk Asus store also write speed was spiky it would ride a bunch then it would slow down then ride a bunch again I also measured power consumption and the pocket Nas uses about 11 watts of power Max about an hour into a 300 gigabyte copy some cache must have filled up because write speed slowed down to 45 megabytes per second and stayed that way until the copy was done you can see that clearly on this network graph the write speed just flat lines after a while but it is a good long while most people aren't copying hundreds of gigabytes at a time like this copying back data from The Nest I got up to 300 megabytes per second that's about three times faster but it didn't quite saturate the 2.5 gig connection still that's a lot faster than I could get with the Raspberry Pi and the read speeds were a little bursty still so write performance is certainly better than the pi but not as much as I'd hoped and read performance is very good but it still couldn't saturate my 2.5 gigabit Network I like the promise of this little pocket Nest but it's not without its compromises besides the origami approach that makes this thing harder to assemble the only other negative for me is this little fan and heatsink for a prototype it works but even with a fan control on to slow it down it's a little bit noisy I hope the final design has better cooling now moving on to this PS2 style slab of black plastic this is The Flash door 12 Pro it has a quad core Intel n5105 CPU it ships with four gigs of RAM which in my opinion is a little too small eight would probably be adequate but I upgraded this one to 16 gigs which is the maximum the n5105 supports since this thing uses an Intel x86 processor I could even install trueness on it if I want and I will we'll get to that but for now I'm going to test it with ADM which is aces store's basic nasos based on BusyBox Linux oh and did I mention I also filled up the whole thing with not six but 12 one terabyte ssds I bought these team group nvmes with dram cache so hopefully with this many they won't slow down after an hour like the pocket NASA and it's cheaper SATA drives and one other quick note I love Aces store's tool as m.2 slots all you have to do is pull back on the tab and push down on the drive the nvme ssds I bought were five bucks more more per Drive than the SATA ones in the pocket Nest so all in this setup costs about 1300 bucks you could keep it under a thousand bucks if you don't go insane filling up all the Bays right away so this thing's double the price of the pocketness but that's also with double the storage plus 10 gig ethernet and 16 gigs of RAM so it's definitely an upgrade in specs for double the price does performance also double short answer yes the little Celeron CPU inside does bottleneck write speeds a little topping out around 600 megabytes per second but that's still five times faster than the pocket Nas and read speeds are flat out on this thing I got a solid 1.2 gigabytes per second all day but performance isn't the only thing this box has going for it the out of the box experience buying a pre-built Nest is night and day sure you can't tweak every little knob in detail but the software is easy and intuitive on first Boot It upgraded itself to the latest firmware the setup wizard holds your hand just enough to get a storage volume going in less than five minutes and after some raid 5 testing switching to a 10 Drive and 2 spare RAID 10 array only took a minute or so but both raid setups still saw write speeds averaging around 600 megabytes per second and read speeds still saturated the 10 gig connection at 1.2 gigabytes per second regardless of the raid type I even tried a three derived RAID 0 and that too wrote to the drives at 600 megabytes per second so the limitation is in the CPU here and there are two reasons for that first the CPU is slightly underpowered for a bunch of fast nvme drives but more importantly it doesn't have enough native pcie Lanes it only has eight lanes and if your stuffing drives in all 12 slots you'd need 48 Lanes to give them all the bandwidth Please add on the 10 gig Nick and you need more than 50 pcie Lanes to avoid any bottlenecks but then how does this work 50 Lanes of hardware on 8 Lanes of PCI Express well Asus store is pulling a little magic using these tiny as media chips they're like ethernet switches but for PCI Express devices they split it up one lane into many so these chips split up eight Lanes into all the lanes required for all these drives but switches aren't magic the overhead takes out some performance and the result is a Nas that can only really ride around 600 megabytes per second no matter what type of raid you use and that's not bad it's just I see the potential and I want more maybe in the next gen flash store read speeds were great though and that's more important for what I do video editing over the network I'm going to replace my old edit Nas with this thing the nvme drives in here should be a little faster than the SATA drives I used in that one plus this Nas only uses about 50 watts of power under load so it'll save a tiny bit on power consumption over the Xeon system I was using to test video editing performance I copied all 13 hours of footage from my Mr Beast project into a Final Cut Pro timeline besides crashing once when I tried copying like 100 gigabytes at once the nas handled editing over the network like a champ the crash was more Final Cut Pro's fault than the nas just as an aside to get the best experience on my Mac I turned on SMB multi-channel and the Mac OS VFS compatibility checkbox in the isostore SMB settings but nothing stuttered at all while I was scrubbing around and I'm playing the full res 4K Clips here I was a little concerned about heat since there aren't any SSD heatsinks the design relies just on this one fan on the bottom but the CPU stayed under 85 degrees Celsius most of the time and the drives never got over 50 degrees in my testing the Box puts out a little heat on the left side so on a cold day I guess I could just use it as a hand warmer and the fan is audible but it isn't annoying like the tiny fan on the pocket mass and if you put it under your desk or in a rack you're not going to hear it at all so what's the verdict does this thing win well we'll get to that first I wanted to highlight my favorite new feature on the flash store and that's trueness wait what I've always wanted ZFS on my Asus store nasas but my older boxes only run ADM and right now at least ADM doesn't support ZFS only basic raid and BTR those aren't bad but ZFS has features I like for data integrity and sync so when I found out Asus stores not locking down this thing at all I had to see how true Nas runs this box is a pretty standard Intel PC under the hood so you can press f2 during Boot and get into the bios so I did that and installed trueness on an external SSD I have a detailed write-up on my blog so I won't go through how I did it here check out the link below but bottom line it was easy to get trunass up and running just like with raid ZFS write speeds always tapped out around 600 megabytes per second but it was a little strange I could only get around 700 megabytes per second read speeds using raid Z2 and even if I used a ZFS stripe I could only get about a gigabyte per second which was slower than the 1.2 gigabytes per second I got with ADM the CPU did seem to run a little hotter under trueness though so I'm wondering if maybe thermal performance or fan control tuning could have made a difference I checked on the CPU frequencies and it looked pretty normal bursting between two points 6 and 2.9 gigahertz but it's something I'll be testing more for now I've switched back to ADM but I'm floored that Asus store is making their Hardware more appealing by not locking it to their own proprietary OS in 10 years if flash store support ends I could still install another OS on it and get more useful life out of it that means this Hardware is infinitely more flexible than a locked down box of course the arm board that the pocket Nest uses was never locked down at all except for the fact that a lot of armchip fenders like rock tip never really provide long-term Linux support for their silicon so this rock 5 is faster than a Raspberry Pi but it's also nowhere near the speed I got with a flash door even considering it's half the price I'm excited though because this pocket Nas that Rick made was just his first try makers like Rick keep iterating and there are a ton of cool DIY Nest solutions that are out on the horizon even so not everyone needs a 10 gig video editing and ass like I do the pocket Nest would be awesome for travel it runs on 10 watts so it's going going to get a lot further for you off grid but in the end I do have to Crown a Victor this round once again goes to the Asus store it's the one that I'm going to be installing in my rack but Aces store better stay on their toes what if I turn my monster arm server into an S it's 128 Lanes of PCI Express bandwidth would eat the flash store for lunch I mean it's more than double the price of the flash store but if I cared about fairness I wouldn't be running a Raspberry Pi against a consumer Nas no would I so we have a Victor for round three but who are these nasas for all flash Nas devices serve a pretty small Niche right now ssds are still too expensive to beat hard drives on a price per terabyte basis and hard drives aren't that bad even for video editing if you throw a bunch of them together in raid but flash is the future I think both of these guys offer a glimpse at the type of size performance and silence we'll expect out of our storage in the next 5 or 10 years for all but the most devoted data hoarders these things will be the norm not the exception they just need a little better CPU and a little more time for SSD costs to come down until next time I'm Jeff gearling

Channel: Jeff Geerling

Views: 379,708

Keywords: asustor, flashstor, ssd, nas, edit, pocketnas, rock 5, model b, raspberry pi, storage, homelab, enthusiast, speed, 10g, 10 gbe, gigabit, network, 2.5g, iperf, performance, copy, smb, multichannel, truenas, right to repair, freedom, os, ubuntu, install, tianocore, bios, uefi, intel, n5105, nvme, sata, attached, jellyfin, final cut, media, server, efficient, premiere, adobe, portable, travel, arm, rockchip, rk3588, diy, maker

Id: 5QH8Dj6g_Nk

Length: 12min 58sec (778 seconds)

Published: Mon Jul 10 2023