What did you DO?? - The "Jellyfish Fryer" All-SSD Server

recently it was brought to my attention that some of my friends in the industry went out looking for a high-speed server for storing and editing their high-res video without checking in with me now I am far from the lone authority when it comes to this stuff so it's not like it's impossible to build something really awesome without my help snazzy labs managed it just fine it's just also pretty easy to get completely taken for a ride and when I saw how much I just een mkbhd and Jonathan from TLD paid for their jellyfish well I wasn't mad I was just hurt and disappointed I mean guys here's the thing with the exception of TLD actually didn't built in anything but for the others of you I put together reliable mass storage for you because that was what you said you wanted if you had just told me you want high performance at any cost come on guys it's me I got your back so I've channeled all of that frustration into today's video we're in collaboration with our special guest Patrick from serve the home great site by the way worth checking out after the video served the home I'll be creating what I call the jellyfish fryer a faster more reliable and just plain better solution summer break just started but it's never too early to start looking for a new laptop for back-to-school every words in PC laptop is ready for whatever games or school projects you're going to tackle and you can check out their special deals at the link below [Music] so yeah guys those numbers you saw at the beginning of this video that was no typo that was correct a 120 terabyte jellyfish costs its lucky owner about thirty five thousand US dollars and that's assuming that they don't require luma forages vague performance boost option or more than a handful of video editors connected to it but the thing is it gets even crazier than that not only does a 120 terabyte jellyfish not actually have a hundred and twenty terabytes of usable capacity it's closer to 80 by the time you account for the storage space that's lost to redundancy and the about 10% of extra reserved space that luma Forge has left to ensure that ZFS functions correctly now here's the thing this pricing might not offend me so much if there was something magical about the jellyfish like I don't know if it used intel ruler nvme SSDs and it fit into like a a ruggedized case that you could throw on your back as you're slogging through the jungle on an on-location shoot or something like that but it's not special and to put it in context not only was I able to kludge together a solution that matches their pricing I was able to do it while still overpaying for my server the 45 drives tornado is not cheap and I outfitted it with all solid-state drives instead of mechanical ones at the same usable capacity and you don't even have to be a tech expert to build something like this because we went with an off-the-shelf bare-bones server most of the work was done all I had to do was spend about 15 minutes shucking the drives from their packaging and popping them in here I did a separate video and with 27 of these 3.8 4 terabyte iron wolf 110 nas SSDs I've got over a hundred terabytes of raw solid-state storage ready to go so from there all that was left to do was let the phone-a-friend to show us the right way to configure it because is it correct to say that configuring an SSD based nas is not the same as a hard drive based mass that's a hundred percent true alright so you want to introduce yourself then sure hey guys I'm Patrick from serve the home we started s th ten years ago building sousou's just like this and now we review server storage and networking hardware all the way from scrappy DIY eBay stuff all the way up to high-end six-figure plus enterprise gear so I kind of assumed that we would be using freeness but in addition to that you came with a bit of a trick up your sleeve right I did ok why don't we begin though with the basic install sure okay so FreeNAS installer super easy it actually automatically does this so you don't really need to do anything and you get some really kind of old looking ass yeah I feel like I'm like gaming 30 years ago so what we'll do is we're gonna select two drives for our boot drives so these are the 250 gig SSDs that shipped with our store NATO they are and then we're gonna go fancy and we're gonna do a fresh install we're good to put a password in sir sure make it easy password serve the home now that we're booted up and we're in the GUI we can start to configure our server and yes it's more complicated than a jellyfish but I mean I think it's fair to say we're we at like 50 clicks listen less than that it's really not that hard now we'll go into a little bit more detail about how we're configuring this drives and Patrick will probably talk about it in a blog post later on in the video but for now once this is done the big thing we need to do is apply that special script which is going to do a couple of things number one is it's going to apply some SSD specific optimizations that FreeNAS intends to include in the next version of their software but that for now they've back ported into this version for us to do this video the other thing it's going to do is it's going to change the way that these four ten gigabit network ports on the back function to be more like a jellyfish so this is basically sacrilegious in the networking world but the way we want this to work is such that if I were to take a network cable plug it into the back of my nest and then plug the other end into let's say like a thunderbolt 10 gig network adapter and a laptop I could just connect directly to this thing like it's a big old gigantic high-performance USB Drive now that's not the typical way you would do things no it's definitely not and what was the reaction you got from industry professionals when you asked them to do this well my first reaction when we talked about it was I said wait you want to do what so for 20 plus years the industry has said no use switches and those switches allow you to scale out and use more clients you have can have more security features I mean it's it is the way that you handle this type of situation when you have a high performance storage of multiple clients so I'm gonna give the counter-argument to that because the way that I see it when you're dealing with a techie person or a sysadmin type person on site that's managing let's say a video production sure switches are the way to go no doubt about it but if you've got people that are accustomed to just plugging in an external drive and working off of their laptop well I'd say that it's not crazy to imagine that it's far lower friction to say look it works exactly the way you're used to you plug in one cable you plug in one cable and you have access to your storage except instead of a Thunderbolt cable you use this weird thing so that's it we've got a healthy twenty six drive raid Z 2 with a hot spare so in the event that any one of those drives fails it starts rebuilding it immediately and you might notice that capacity looks a little low seventy-seven point nine one TI B's so those are Tibbie bytes if we convert that to terabytes that's actually eighty five point six terabytes so we are coming in right around the same capacity as the jelly-fish for actually a slightly lower price because we're like $900 less or something like that okay we should be ready to connect directly to the back of our nas here transfer some files do some video editing Wow I can look at that even with the 8 core core i9 you still spike them when you're just opening up a project like this and well we we peaked at around two and a half gigabit per second so scrubbing is actually not that demanding as we go through this 8k footage here so we're sitting right in the neighborhood of one and a half to two gigabit per second while playing back it looks like it settled in a little bit lower than that let's speed up our playback that's gonna hurt you okay so it's not a terribly impressive demo because it's just one 4k stream but the NAS works I think we need to kick this up a notch we do so we just completed our twelve machine Land Center these are all rise in 2700 X's they've all got ten gig lan and we're gonna grab ourselves a few quad 10 gigabit NICs connect to all of them at the same time and see just how many video streams we can do alright so there it is for ethernet cables and theoretically we fire this up and we have access to it we we fire this up well that'll help we fire this up and we'll have access to it so we've got three links running through our switch down there that are in an LACP group so that's basically link aggregation so that's for a total of 40 gigabit plus 40 plus 40 so that would be 120 gigabit so that is effectively the same as if we had three of those quad ports and they were just plugged directly into the back of the computers then all of our machines are plugged into the switch and we've loaded up Adobe premier trials on each one of these machines with the intention of hitting all twelve of them at once and seeing just how many 4k video streams our server can handle so Jake oh you're already on it look at him go these young kids today they got so much energy you know so just as a recap here guys this is 16 MPEG 2 files that were encoded at 100 megabit per second so that would be comparable to what you would record in the 4k high-quality preset on something like an a7 s2 now you can see here that this is a freaking lot of bandwidth especially when we have a lot of motion across all the clips and we can spike as high as 4 to even five gigabit per second and that's while we are just playing back nevermind if we want to scrub through the footage if we scrub through the footage we are actually going to be able to saturate our 10 gigabit link here so you can see I'm sitting in the 9 to 9 and 1/2 range isn't that absolutely beautiful but performance is still smooth this is actually a shockingly good editing experience this is it moment of truth you ready are you just doing that one now I'm gonna do all of them no you ready yeah okay one two oh crap this one's at one-quarter res oh they're all at one-quarter res dang it no one quarter res is not enough for us okay changing the res starts stars so all six of mine are going are we dropping frames or is it smooth looks pretty damn mean pretty smooth to me okay so I got our overall feed here nice nice how we doing so casually Oh twenty five gigabit twenty five gigabit I think we can do better so one challenge that we've got is that while we'd love to just throw more video streams at each one of these machines we're up against a technical limit within Adobe Premiere where it only allows sixteen sources for this multicam set up and also up against a CPU limits so our CPU is actually sitting anywhere from around 70 up to 90 percent as we're going through and playing back all these files so clearly we can't just throw more streams per machine but what we can do to hit our network harder is scrub all the machines so I think we need some helpers okay so thank you gentlemen the test here is simple we are trying to suck as much bandwidth as we possibly can from the video server sitting on this chair over here now we're playing back 16 100 megabit streams per machine as it is but that's not enough that's only about 25 to 27 gigabit child's play so if we scrub through the footage we can get each of these machines to pull about nine to nine and a half gigabit and we want to see just how hard we can make it hurt okay so I'm peaking at around four gigabit per second on each of mine and we're looking at about 40 gigabit overall so I think our aggregation is not working not working all right so the good news is we managed to play 184 4k video streams off of our server and we managed even to scrub around through them during that process the bad news is that it looks like unless we can get link aggregation working properly it would in fact be better to go straight out with ten gigabit connections these are the computers I just didn't feel like spending the couple thousand dollars on a bunch of those quad 10 gig cards so we're gonna call this we're gonna call this good enough for now so that's a pretty cool demo if you're into editing video off of a shared storage resource and it's actually pretty similar to the one that luma forge used when they launched the jellyfish just bigger and badder because it's all flash and somehow at the same price but we haven't even addressed some of the other really cool things about this configuration yet so luma forge is using two striped raid z2s so that's kind of like raid 60 so a stripe of raid sixes and that means that they can suffer anywhere from two to four physical drive failures before they lose any data now we can only take two Drive failures and then a third assuming that one of them manages to rebuild with our hot spare so one two three over time that's kind of similar I guess but the story is a little bit more nuanced than that so Patrick you guys have a calculator that people can use to calculate what the expected time to data loss is for a particular configuration and what did we come out at with this one right here yeah this one we basically had less than a point zero two percent chance to have it fail over the next ten years so we think it's pretty reliable and part of that is really just the fact that the hard drives are a lot slower than the SSDs when it comes to rebuild which means that even if a drive fails your array spends less time in a degraded and vulnerable State thing if you're using hard drives a couple other advantages of SSDs are that these ones are about a hundred times less likely to read a bit incorrectly compared to even reliable hard drives and they've got some other practical advantages too no vibration no noise and if you're on the go like you're you know if you were to put some like mechanical hard drives on the back of a Jeep in the jungle for an on-location shoot that is a super bad idea and this wouldn't be nearly as vulnerable to those kinds of vibrations which by the way yes do kill drives you were telling a story about Yahoo a little while ago oh yeah Yahoo a long time ago actually put drives on storage arrays that were going across the parking lot and one of things they found was that when they got the system's powered up and the drives powered up on the other end they had a big problem with Drive failures and the reason for that was that those carts were actually using solid casters and that cost so much vibration that the hard drives ended up failing so it's like a good example of why hard drives even though they're generally reliable even you know if they get a little bit shaken up while they're powered off a lot less reliable than solid-state drives in that scenario so we're not actually necessarily done yet we had some further optimization ideas I mean Patrick really really wants to use a switch because as we saw unless you have a ton of streams coming into a single machine you're nowhere near 10 gigabit needed for each individual station but that's gonna come down to you know how the user wants to use it for now we were trying to just go after well here you can have this but DIY we could also get better performance and lower power consumption by using more modern CPUs caching was something that we talked about and it's something that luma Ford actually offers as a performance boost option for the jellyfish they add an opt-in right cache we've some more RAM and then we think may be faster CPUs they wouldn't confirm that for me for an extra thirty five hundred dollars but you recommended against using something like an octane right cache for an all SSD based machine can you explain why that is yeah so you tend to use caching drives especially when you're going to hard drive in the idea was you have something that's faster than hard drives that's able to absorb random reads random writes and then smooth it out so it's more sequential and better for the way that hard drives like to be accessed but solid-state drives are different than hard drives and because of that you tend not to put as much caching in front of solid-state erase and so in this case we just didn't think that we needed it all right so we maybe aren't quite done but that is it for today but the sort of wrap-up for this video is a bit long but it's also really important so try not to tune out guys we did this video for a couple of reasons one was to demystify some of the marketing mumbo-jumbo that goes on with solutions like the jelly-fish compatible with Mac Linux PC what does that even mean of course it is it's a commodity nas running free open-source software but the other reason was to acknowledge that while we do prefer to do things our way that doesn't mean that our way is right for everyone and in fact even between the two of us and IX systems support staff we still ran into a few hiccups today so then I actually reached out to marques mkbhd to send over a couple short clips explaining why the jellyfish has been great for him stop Lynas mkbhd here so there were a couple things that the jellyfish did specifically in the mkbhd studio to make our lives easier this set up being one of the biggest ones so this thing was out the box and operational within 90 seconds like it had all the drives installed already everything was fresh out the box all we really had to do was install the jellyfish app on every computer that we wanted it to be on that's three in my case plug it in hit the on button and everything set itself up and then we actually had some assistance from the luma forge team as far as setting it up to make sure it would work perfectly with Final Cut Pro projects so workflow tips essentially so they definitely went above and beyond with that but really truly the beauty of it was going from not having any solution for working on videos at the same time to the next day having one and being able to edit you know close a project on one machine open it up on the other and keep going that was something we were able to do very quickly and seamlessly because the jellyfish I think at the end of the day when it comes to spending a bit more for a better overall solution for us then better hardware it can look a lot like the car enthusiast market like some people will buy a certain type of car because they know that will be a great tuner and they'll be able to get more horsepower out of it and work on it and get better numbers but they know that's that's more time they're gonna spend because that's their hobby but some people just want to drive to work every day and back and grocery shopping and pick up the kids dance just needs to be reliable and they will spend more on a car that's new just to do that and they can go to the dealer if something breaks down for convenience to fix it and that may seem lazy to the first person but that's just the way it is that's what the jellyfish is so I am fully aware that there are buildable solutions out there that will do better numbers and better read/write speeds and things like that probably even a better looking box than the jellyfish but we've been too busy editing videos off of it to think about any of that so that's where we're coming from so in defense of luma forage then it's easy to use when I called them they picked up the phone immediately they offer support plan with next day replacement Drive so that two years down the road you don't have to remember all the IPS of everything and how to do anything like companies like this do need to charge a premium so they can continue to provide support they offer a complete package so you can just forget about it like you're not buying a box you're buying a solution and in fairness to them the hardware that they're shipping is not actual garbage it's not one of those things where they're charging an enterprise price for you know like a sticker that they applied to something consumer grade like they're using ZFS they're using Santo LS this is truly good stuff their connector and manager is really simple to use if you have to set up a new share or something there's no command line required also here's a crazy thing if you go to a major Tier one storage vendor would you say that the pricing of the jellyfish is unheard of it's perfectly in line with what you would see from the vendor and they usually have great margins and they provide that solution and great support furthermore another thing people don't consider and I talked about this recently on LAN show but in the film and production industry this kind of pricing is not about the hardware that you're getting it's more about the value that it provides like the of money that you're spending might be outrageous but given the amount of money that you will make if it works correctly and the amount of money you might lose if it doesn't work correctly maybe it's not that outrageous after all now we aren't gonna be doing it that way as long as building our own machine is still possible but hey I felt like I had to say it anyway speaking of things I have to say the neighbors app by ring yep that's the same ring that makes video doorbells is all about helping you stop feeling like you're alone when it comes to your security it allows you to receive real-time information on what's happening around you from other users so you can put out an alert if you see something suspicious like a possible burglar you can send a notice if you lost something valuable like your pet and you can even give or receive a warning if a disaster is coming your way like a fire or a flood ring partners up with local law enforcement to stop crime sprees and right now it's available in the US only but more countries are coming soon users have full control of what information is made publicly available to protect your privacy from both other users and law enforcement and it's free to use and download it's available for Android and iOS today so go check it out so thanks for watching guys massive thanks to Patrick from serve the home guys go check it out go buy one of these tracker hats on their merch store you guys have emerged start at worse you don't even get even sell them no these are so this is like exclusive alright so thanks to you guys for watching thanks to Patrick for coming out here if you guys like this video hit that like button get subscribed or maybe consider checking out where to buy the stuff we featured in the video description we were really happy with the store Nadeau as well as our iron wolf Nazz SSDs we're gonna have links to those down there also linked down there is our merch store which has cool shirts like this remember hard drives remember like this as well as our Linus tech tips stealth itty all right finally yeah go join our forum that's down there too

Channel: Linus Tech Tips

Views: 1,741,712

Rating: 4.9234252 out of 5

Keywords: server, transcoding, editing, storage, ssd, space, GB, jellyfish, seagate, ironwolf, videos, iJustine, Jonathan Morrison, fast, Marques Brownlee, MKBHD

Id: zeAce9pofvk

Length: 25min 16sec (1516 seconds)

Published: Tue Jul 30 2019

Reddit Comments

[u/BubiBalboa]

It's honestly pretty sad that one of the very few mainstream Youtube channels which regularly produces well made datahorder content is so disliked here.

I'm sure a large percent of the 150k readers of this subreddit would enjoy this video. But they can't because the first few people decided this video isn't worthy of this subreddit.

If you think the guy is talking bullshit maybe use this thread as a teaching opportunity to enlighten the less educated among us.

[u/Swizzdoc]

His vids are awesome and informative. Everything is slightly OTT but that‘s part of the fun. It‘s a show with informative content.

I‘m not likely to buy any of the extravagant stuff, but I recently bought the G613 because of bis reviews. So, thanks Linus.

[u/5footbanana]

When the original videos came out the storinator was supposed to replace people hoarding external drives which it clearly does very well.

The lumaforge is fast enough to read/write directly which is different use cases. Both items would be used ideally for backups.

Also companies make money from their USP, not saving a few grand then having to troubleshoot unraid and wait around while data copies around.

[u/floriplum]

I haven't watched the video but in the article to the build they mentioned a 26 disk raidz2 array with one hot spare. Seems like a great idea :)

PS: since i don't know if they actually store data there or just edit a video and move it to another storage i don't know if it is that risky.