What Can a $60,000 Media Server Do?

four years ago I built a server with old used Enterprise Hardware that I scrapped from eBay excluding the rather expensive 336 terabytes of storage the rest of the system cost me less than three thousand dollars in the server world that's nothing and true Nas core formerly known as freenas was the only reason it worked it was the glue that held everything together because not only was my Hardware from different Generations but it wasn't ever designed to even work with one another but trueness held it all together an open source BSD based operating system for file storage that's almost platform agnostic works with darn near anything and the performance of this completed server running trunes was amazing so good that we actually lovingly named the server Margaret and Margaret served us well for years but I do have one small confession to make for the last year fan editing projects off portable ssds without backups and once a video is completed while we try to move the footage to a reliable but glacially slow consumer net it's a terrible system but it's one that we've adopted because our old equipment because Margaret was too loud and too hot and used too much power for our small downtown office that was something that wasn't a problem at our old office which was a warehouse with huge uninsulated areas and Industrial energy rates but then something neat happened IX systems creators of trunes reached out and said hey we want to send you one of our amazing m-series Enterprise systems I was excited obviously but I was also kind of worried that I wouldn't be able to drive them sales with my audience and well they told me something to the effect of and they said it nicely but something to the effect of yeah duh the server we want to send you is fifty thousand dollars anybody who's shopping for something like that kinda already knows who we are we don't really need your help so then I was like okay but then what do you need me for and they told me that the only purpose in sending me this insane server was to entertain and educate both you and me and it's my hope that by the end of this video you'll be inspired enough to get a slice of this insane performance for free using their open source software in your own home and that's pretty awesome pitch so I told them yeah send me the server and here it is it's been here since May of 2022. sorry IX systems it's been here on the floor because it is super super heavy 145 pounds Benjamin had to push this up the stairs by himself and he complained to me about how his back hurt and how it was hard but team lift stickers come on I'm going to do this all by myself right here so start with no actually we might and then because this thing is ZFS there are a number of tiers of storage speeds and this is kind of the Bottom Rung this is as slow as it gets we have got at the very bottom end a bunch of 18 terabyte SAS hard drives so these are Western Digital drives 18 terabytes it's nothing to scoff at and then above that as the other fraction of our pool we have a bunch of seven ssds these are also SAS all of these Drives By the way are what's called dual Port SAS and allows them to communicate with both controllers at the same time that'll make sense in a second but these are going to be even faster than our hard drives and when you pool them together when there are many in quantity things get even faster but this is not even really the realm of speed this is just the means by which we store everything on the server itself so let's get to the server itself some rails that'll be nice because this thing's going to be quite heavy we've got I bet this is power cables power cables and then we've got the server itself this thing is really light oh my gosh okay that's kind of heavy foreign so as mentioned previously this shelf holds 24 discs but we don't actually need all of that space so we have here a few blanks we are only going to be using 18 discs and in the future when we need to expand we can all the fun stuff is here in the back we have two controllers these are identical machines identical computers and the point is that if and when one of them fails there's a failover it can immediately pick up where one of them left off There's No Interruption there's no downtime which costs money for big businesses for us maybe a little unnecessary but that's how Troon has builds out their servers we have redundant power supplies there are dual 1200 watt dual 1200 watt power supplies they're so tiny and cute like all power supplies and this fan is going to be really really really really really loud but these are 80 plus Platinum rated we've got two and of course that's for redundancy in the case that one fails it immediately switches over there's no downtime and you can keep your stuff running while you wait for parts but ideally you have one or two of these on hand not that they're known to fail by the way but you know just in case and then over here we've got a beautiful fan and this is where some of the money is a lot of the money actually if we pop these two boys open oh that one's blank aha this is known as our L2 Arc or our read cache this is a kioxia um u.2 nvme Drive of some capacity one terabyte and we will explain the importance of the read cache a little bit later but I think the best way to kind of understand what's going on is to pull out one of the controllers we don't need to pull out both of them because again they're the same they are identical we have two of them inside here so there's a thumb screw that you definitely need to turn before you try and rip it out of the machine oh oh this thing is a thing of beauty since again this other controller is identical we'll just put it to the side here this is where the good stuff is now let's start by actually looking at the back I O here we've got an IPM Management Port by ethernet we've got a bunch of USBS we've got two 10 gig nics that are actually on the motherboard the chipsets are on here which is great we don't need any extra network interface cards because we can just use the ones here on the board we don't have enough overhead to require more than that and then there's a couple PCA cards that we're going to talk about in a minute but the thing that's most shocking to me is that it's just an ATX motherboard a lot of servers will use weird form factors and custom sizes and proprietary nonsense no this is just a super micro X11 SPI TF ATX motherboard it kind of looks like one you would have in your standard computer but this thing is not your standard computer let's start with the CPU the CPU is actually nothing all that's special it's an Intel Xeon cascade-like Processor it's a 10 core 20 thread CPU that has a base clock of 2.4 gigahertz and can boost clock up to 3.2 which is by these days and standards not insane but it's certainly sufficient for anything that we're going to be using it for it'll actually even be powerful enough that we can run virtual machines and a couple of containers it's Docker container it's it's going to be great this is a 100 watt tdp's chip though it's very warm and you can see that the CPU Cooler is a lot smaller than one you might find in your desktop computer and that's because of these bad boys fans static pressure fans move a crap ton of air they do it at the expense of noise these are loud but they save you space when you're in a Data Center and you're stacking a bunch of these systems up square footage is super valuable and who cares if it's kind of loud so that's what they choose to do in these situations other things that you'll see that are pretty conventional on a standard computer well you've got a 250 gig nvme WD blue SSD those are found for like 50 60 bucks if even that so why such a cheap SSD well because the operating system actually mostly loads and lives inside of memory you really only use this when you're booting and shutting down and a couple of other instances and so you don't need a super fancy pants boot Drive that's pretty much where the similarities with your standard computer will end however let's start by explaining how bananas this thing is by talking about memory we have got six dims that are 32 gigs a piece which gives us that's right 192 gigabytes of ddr4 ECC error correcting memory that's per controller so in total we have almost 400 gigabytes of memory and then you'll see this bizarre Ram stick with a cable coming out of it what is that well it's something you're never going to see in your standard computer but it's something that's becoming very very very popular in applications like this it's called NV dim or non-volatile dims dim stand for integrated memory module dynamic it doesn't matter it's non-violence volatile so standard Ram when you lose power or when your system crashes all of that data is gone that's bad in applications where you're storing a lot of stuff in data these things are designed to be crazy fast so a lot of what you're actually reading and writing from to the server is stored in memory until it can be offloaded to slower areas that right cache needs to be written too quickly where do you put it though if you can't let that stuff get lost in something like this this is a 16 gigabyte module that has this cable that actually connects to a bank of super capacitors and in the event of a power loss or failure it writes everything that's stored in it to your ssds so that nothing gets lost additionally this little controller right here this card interlinks the two systems these two controllers and these NV dim modules are consistently talking back and forth with one another making sure that they have data parity so you have multiple redundancy situations it's really freaking cool and then well we move over to here and this is kind of just a you know standard PCI slots and there's not much crazy stuff going on here so this first card right here uh connects our U dot to nvme express drives to the controller so this just this talks to this that's it and it's basically an extension cord these are actually nvme but they have a different kind of you know you can't plug these in you got to put them separately so if that's what this does the middle thing this is pretty cool this is an external SAS controller you can actually operate two separate disk shelves with 24 discs each bringing the total system capacity to two petabytes of storage which is bananas but to be able to communicate with those extra separate disks well you need communication ports right here so these are 12 gigabit SAS connections each that can connect to each shelf which is pretty crazy and then this controller here well this connects to the 24 drives that are at the front of the machine so this is for local this is for disk shelves and then again as we mentioned the NV dim controller and that's that's pretty much it I mean there's really nothing to it where this machine is going to thrive is when we actually boot up true Mass so let's do it no we gotta install it first installing this thing was honestly a piece of cake we just snapped our rails in no cage nuts required which is fantastic we slid the server into position that part was kind of hard because it was really heavy and then we plugged it into power we plugged it into our networking switch and we turned it on and that's pretty much it it just worked and I'm not exaggerating because most of the cost is sure the hardware is expensive but most of the cost is in service and support and included in that is not just the option for a year for me to be able to call up IX systems day or night and have them help me right when I wanted but it also includes ready for deployment installations I just plug it in turn it on and it's ready to go it's been pre-configured from the factory to work on my network and that's pretty cool but sometimes you want to get into the admin software to mess around with stuff and get things going so I want to show you what you can do in this software before I show you what the server itself can do speed wise this is our dashboard and it gives us an overview of everything the system is doing or maybe not doing it's hovering at around zero to one percent CPU usage these xeons are pretty overkill for our application but there is one instance that I'll talk about in a minute where you could get them running a little bit now we go to memory there's 191 gigs but hold on there's only 12 gigs free I thought we had 192. where's it all going you can see this is all being stored in the ZFS cache this is basically just retaining all of our most recent files this is why this server is so fast when we're editing videos it's not continuing to pull all of those clips off the SSD and off the SSD enough no no no all of those video files are stored in memory and then in the weird occasion that I do need something from an SSD or from the hard drive pool that's not on uh that's not volatile already I can import that into memory the end result is something that feels rapidly fast all the time because your stuff is already ready to rumble so you can pretty much throw as much memory at true knives as you want and it will just eat all of it up it just retains as much data as possible even if it's redundant technically because you've already got it stored on the pools speaking of redundancy we do have redundancy in our pools we've got hard drive pool and we've got a solid state pool you can see I've got five v-depths so we have uh one spare on our hard drive pool and we've got one spare on our SSD pool and the end result is a bunch of SSD storage we've only used five percent because really all we've got stored there is our current projects and then 60 use space on the hard drive pool that's a little bit Fuller than you might expect because well in addition to every video we've ever made the finished copy that is we've also got a bunch of internal assets and we've got every single video clip we've shot from 2021 until now so as this thing gets more full I've got a couple options number one I can plug in Margaret's dish Shelf with all the old hard drives that I've got 300 terabytes of them and uh just add more space but that will draw more energy it'll create more noise and more heat so what I'm more likely to do is just continue offloading stuff to lto now a server is redundant but it's not a backup you still need backups and so we've been backing up our footage for years to lto and once it gets to the point where we're not likely to really ever need it again I think one or two backups is sufficient so you can actually see I've got a tape plugged in here I can open my project browser and I can see the 2019 folder off of the tape I can see every single video that we exported that year I can click on it and it shows uh oh well you know they're not a current sponsor anymore so ah uh how about this one I can click Intel to arm we've got uh 3.81 gigs stored on the tape for this video and if I want to recover it at any point in time I can that's thanks to the Edith lto Mac Mini Server that we're running in that room as well but back to Doris which is what we've lovingly named this new server you can see that we've got our interface overview next this is an aggregate connection it's called a lag a lan Aggregate and that's because I took two 10 gig connections plugged them both into my switch but then at the switch at the network level I combined them virtually into one connection the end result is a 20 gig connection that both Benjamin and I can hit at 10 gig each on our computers at the same time and it can keep up with the demands of both of our machines if in the future we have more people that are simultaneously editing we can just put in a larger Nick and it will be able to provide 10 gig connections to everyone which is really really cool and then at the top we've got our two controllers we can see that the current one Doris B is the one that is active but I've got high availability enabled this is what I mentioned earlier I can have something happen to this controller it can turn off it could break whatever could happen and instantly this thing will absorb all of the volatile data from this machine so that I don't lose anything at all and then it can instantly pick up where this one left off after just a few seconds in fact it's so fast you don't even realize that anything is down pretty darn cool but let me show you what you can actually do rather than just show you an overview so I'm going to show you how easy it is to create new shares let's go into storage and then we're going to go into pools you can can see we've got our hard drive pool we've got our solid state pool and then underneath here we've got something that are called data sets these are basically just sections of reserved space from our pool that can be used for a specific purpose now currently these all have the same amount of available data because I don't have them a storage limited but let's create an example where I might want that so I'm going to go to hard drive Doris and one application I might want is a time machine share that I can use to back up all of my Macs on our Network automatically overnight to that server so I'm going to add a data set I am going to call it time machine and then I'm going to click submit that was easy now I can go over to sharing I can go over to SMB which is the protocol that we're connecting to the server with even on Macs it's the best way to do it AFP is old NFS is even older SMB is the way to go I can press add and then I just basically have to draw a path back to that data set that I just created so time machine okay and then I'm going to name this time machine sure why not and then in my purpose I'm going to say the purpose of this data set is a multi-user time machine I click that and then I just push submit and then it's going to ask me if I want to reboot my SMB services so I'm going to save and restart this takes just a couple of seconds and then what I can do is just navigate directly into um system preferences let's actually search time machine because I don't know where anything is in the new system preferences app I can press add a backup disk and there it is it's already there time machine on doris.local I can set that up as my backup disk and it will all be ready to rumble pretty cool but let's say that once I start backing up it's using a lot of data because time machine will just keep as much storage as is available I don't want it to take up all 40 terabytes which is what I have of my pool so I'm going to go back into storage I'm going to go to pools I'm going to go to time machine and I'm going to press edit options and then over here quota for this data set and all children so I do have children data sets I'm going to do one Tebby bite okay I push save and then check it out there's only one terabyte available to this specific data set once I've hit one terabyte on our server with all our computers it's just going to start deleting old backups and then making room for new ones but because time machine is versioned one terabyte lasts like months it's a really long time so this is a fantastic solution to just easily and quickly create a new share and a new purpose for your server you can go a lot further than this and I will in the future on another video on how you can make your own true net system but you can go into plugins or into jails this is an area where you can run actively virtualized and or containerized applications on your truenas system to provide services to your whole network so you could create a next Cloud uh uh instance where you can access all of your data on your next Cloud file on the internet like out and about you can also create a Plex server there's so many things you can do and we will talk about that but this thing can get really powerful and does a lot more than I've shown you today but I want to show you how fast this thing is and why this server is really pretty special this is the snazzy Labs workstation a 10 gigabit capable Mac Studio where Benjamin edits all of the videos you see here on snazzy labs to illustrate how Bloody quick Doris is let me show you this a SATA SSD this is what we write all of our footage to from our recorder attached to our camera a couple of years ago this was about as fast as you could expect or want from an SSD about 500 megabytes read and write in each Direction that's not bad but look at what Doris can do we're going to go into select Target drive I'm going to go into Doris I'm going to go to current projects which is our SSD pool and then I'm going to press start and you can see that Mama we are about double the speed 912 megabytes per second not megabits megabytes so that is about uh seven eight gigabits per second uh the reason we're not at 10 is because there's a lot of TCP overhead that's permitting us from going full bore also because our SSD pool is slightly smaller than our hard drive pool we can select our Target drive to Doris and then archived projects which is on our hard drive pool and we can see that our speeds are actually a little bit faster and that is because we have a greater quantity of hard drives the ssds in our server are faster but there's fewer of them versus having a lot of hard drives they can be slower but and mass they're actually quicker so this thing is bloody fast but what does that mean in the real world what does that translate to all right I am here inside of our Apple archive in the vault this is on the hard drive pool and this is where we keep all of footage from old Apple events and stuff like that so that if it ever disappears from the internet we've got it I'm going to take this file copy it and then paste it to my desktop it's 772 megabytes all right here we go you can see that it's pretty quick but it does take a few seconds that's the hard drive pool because again we have spinning platters and spindles that have to find that file but if we go to FCP benchy which is stored on our SSD pool I can copy this file which is 1.5 gigabytes I can paste it and boom it's already there it didn't even show a progress bar just finished this is where the server is cool though if I go back to that Apple archive I'm going to delete this file okay I'm going to copy the exact same file that I just did that took about five or six seconds and paste it again that was way quicker but why well that's because of our ZFS cache it's that all of these files are stored in memory they're pulled from the pool once and then they stay there in Ram until there's no more room and it has to kick them out based on when the last time we used it was where this becomes really important and effective for what we do is when we're actually editing video so I'm going to open a project here called Winter's last whisper this was available this was the video we last published on our Hackintosh and as you can see we've actually already moved it to the spinning hard drive pool this is our slow pool but it kind of doesn't really matter because once the project opens and it's going to take a hot second because Premiere is slower than dog water you'll see that once we've got the media located and loaded up into the timeline it's just as fast as it would feel as if it were on our native computer because all of that footage has moved off of the hard drive into memory and so the end result is once I've opened up this timeline okay I can play and scrub through this in like instantly it feels like I'm editing off of my Max nvme 5 gigabytes per second uh SSD but I'm not I'm editing off of a server over the local area network and not just off the server but off of a server that has all of these files stored on hard drives it's pretty bananas as a last demonstration just to show you how crazy this thing is I'm going to show you how we import our footage from our SSD so I'm going to go into Ninja V these are two a-roll files it's actually the files that you've already seen in this video and you can see on the SSD it takes a couple seconds for them to pop up so I am going to not open those I'm going to copy these I am going to go to our server I'm going to go to current projects I'm going to go to server upgrade a-roll and then I'm going to paste these clips and it says that one of them is already there so I'm going to skip that one we're just going to keep both and we're moving 171 gigabytes in less than five minutes the craziest thing is that's not a limitation of the server that's this drive that we just demoed is half the speed of our server if we had faster ssds this wouldn't be done in five minutes it'd be done in two a 200 gigabyte a-roll file so that illustrates why this thing is so cool for us because we deal with hundreds of gigs of video for every single project I mean if I go back to our Hackintosh project and then click get info you'll see that just this project by itself this one video was almost a terabyte of data that adds up quickly and being able to do this all remotely where he can open it the timeline here on the Mac Pro or the Mac Studio and then I can go over to my computer and say nah let's make this adjustment here and make this adjustment here and I can run the same file I can open the same Premiere Project without ever moving away from my desk that's so so cool that is just the slightest little bit of what Doris can do and I'm so excited over the coming months to show you not what more it can do but also how you can get a slice of that same performance in your own home for a very tiny fraction of the cost maybe with the old Hardware that you've already got laying around so stay tuned for that get subscribed leave your comments down below what you think of Doris and uh how you feel about Margaret being put out to pasture may she rest in peace thank you so much for watching and as always stay snazzy all right [Applause] foreign

Channel: Snazzy Labs

Views: 205,666

Keywords: snazzy labs, quinn nelson, snazzyq, truenas, server, media server, freenas, plex, jellyfin, expensive tech, enterprise, intel xeon, zfs, open zfs, freebsd, home server, edit server, networking, 10gig, 2.5gig, media server build, hard drive, how to, plex server, freenas vs synology, 10gig ethernet, 10 gigabit, server room, server build

Id: d5Pq7QhqHvk

Length: 27min 21sec (1641 seconds)

Published: Wed May 03 2023