Building the Ultimate DIY NAS on AMD, Ft. Level1Techs, Unraid, & ZFS (Part 1/2)

[Music] [Music] [Music] [Music] there's only two people who can fix our storage problem do you think it'll work it's our only option [Music] terabytes now okay good it's here class what am I wearing how did you get here I always travel about telecom rack don't worry about the I have some server parts mmm yeah this will work let's do this before that this video is brought to you by Linode cloud computing we've trusted Linode as our web host since 2012 and recommend it for excellent technical and customer support reliable uptime and clean interface aside from cloud hosting lynda.com recently added GPU hosting plans for machine learning and neural net use built with our TX 6000 GPUs and 10 gigabit per second network speeds they're also starting to deploy epic CPUs in their servers sign up for Linode comm cloud computing with code G at Nexus 20 for a $20 credit or click the link in the description below to visit linda.com slash gamers Nexus 128 cores 256 gigabytes of memory 12 two and a half inch base 11 World Records broken he literally just appeared and officially started the this part of the segment and Wendell's like here's all the stats so yes this is a really cool story Wendell from level one tax is here to help me out and as kind of indicated in the section before this we have a bit of a storage problem and so I thought you know there's really only two people I trust to help me with this there's Linus and then there's Wendell and I thought well Linus is gonna make some crazy really he's good he's gonna get sponsors involved and it'll be awesome and then at the end of it I'm gonna be like hmmm let's go with the garbage yeah yeah let's what can we do with what can we do with worst part how can we take what Linus would do and and somehow use worst parts but make a really cool server so we got a sort of standard computer case and big air quotes there and then I'm going to be using a sort of standard motherboard but we'll talk about why it's special later and then we're pulling a bunch of excess cooling parts and I've got some old drives we'll pull out I think you have some cool old things we can use as well so we're gonna make this a really fun project and interesting and I guess first though we should talk about this option I think this is what you actually brought for me right yeah it's a 128 cores and it's probably a little overkill I mean like this is this is like the real enterprise you get the stuff online and you look at like real server great hardware I mean it's at the IPMI this one's configured for like GPUs so you can you can put GPUs in here and it's configured for three-and-a-half inch drives there are options for two-and-a-half inch drives of course but it's the double epoch realm server so these are brand new server CPUs from from AMD yeah and you have to take this back with you unfortunately for me but I did want to point out for people who aren't too familiar with level 1 text content you work with enterprise stuff all the time it doesn't always make it to a video but I try yeah yes oh so you've got got the experience and a lot of other cool enterprise hardware that you've brought along will have separate videos on so like this thing if you want to see more about the is it's a dual so it's 128 cores 256 256 threads so if you want to see more on this specific thing we're gonna have a separate video on that where we kind of will turn it on and we're not gonna be able to talk for the first few seconds when it's turned on because it's really loud like I complain about the blower coolers on reference cards and can't really do that anymore 2200 Watts 80 + platinum in this tiny little form factor yeah it's it's in Pratt and Austin can we show the fan yeah so there's the there's the fan right there it's only the one fan it's quite loud this little phase out of 40 yeah it's like it moves as much air is like a 140 so it's gonna be loud yeah so this will have a separate video but that is something I wanted to start off with and say you know Wendell's got some some experience in this stuff and when I emailed you my problem was we're running out of storage we use NASA's but I'd really like to do something properly for us yeah and so you had a whole lot of options yeah you don't want to just plug a hard drive into the network I think that you can like the thing that's really awesome about this is because you've got so many cores you can run a bunch of virtual machines and storage and you can pick this one box and slice it up a bunch of different ways but just because you're using a mix of consumer and let's call it prosumer gear and a box like this doesn't mean that you can't benefit from this kind of technology in a box like this right yeah and that's will show a lot of that with the motherboard today it's an as rock is it as rock rack that group as Rach Rach it's a $500 server board I guess which by consumer standards is an expensive board bye-bye business expense standards 500 not terrible if you're gonna be using it for years yeah so that's got a really cool interface will show later and then this is the box I really wanted to use for this so this is a Silverstone cs3 81 and I have liked this thing since I saw it at CES and a prototype and actually I think I was the only one who who really liked it a lot when it was in prototype phase because silverstone originally offered to send us the prototype because they didn't want to miss the opportunity when we were building a server but we waited for the final one so this is a production model it's really good from a sort of consumer chassis aspect of being able to get a lot of storage it's got some some kind of consumer slimmed down versions of I mean enterprise plug-and-play drives but the only downside with this when you're dealing with mechanical hard drives eventually the mechanical hard drives will die and then it will be worth the price of admission to get those bays yes right yeah we were talking about that with a standard ATX chassis or something yeah it's just oh there's a stack of hard drives it's like this which one died I don't know and also you get the fun of routing the cables again stuff like that so yeah the only downside is this is 8 to 10 ish drives depending on I guess I have to look at it closer but I think it's 8 to 10 drives it's definitely 8 in the second segment with 2 SSD options and so I am concerned we might run out of space at some point well but we've got some ideas for that I mean I think well Oh actually wait where did where did that come from how did you get this is 24 more drives for that okay so if you find that eight drives is not enough we can just spill over into this thing and oh look it's already half populated with 12 more Drive point out that the comments are gonna be freaking out that you've just put this thing on top of that big server it's fine this design for a rack these things are supposed to be like 42 you stack yeah it's totally okay okay so how many is it it's twelve okay well it's a total 24 but twelve bays are already filled got it so you've got room for 12 plus 4 more drives this chassis is going to support a total of 32 three and a half inch drives and up to two two and a half inch SSDs plus nvme storage and so this is a disc shelf is that the correct terminology yep disc shelf and a disc shelf has no it's got no computer in it unlike this so what needs to be controlled by something externally yep and these use of these came out of originally what was it was a yeah IBM NetApp so normally you would have a huge stack of these in a full 40 to you case and that's you know 144 mechanical drive spend spindles plus SSDs plus two servers to control it so this in addition to just being like so your interface here each Drive plugs directly into the controller but with this there is a sort of intermediate controller here and it will take the bandwidth and balance it because you think about serial ata you know serial ata is up to 600 megabytes per second but there's not a mechanical hard drive on earth that's 600 megabytes per second when you've got this mini mechanical hard drives it'll take all of those individual connections aggregating them and then send them down a SAS pipe so that you can take advantage of the full aggregate bandwidth of all of those interfaces when you got this many spindles and when you got this many spindles you can clear a gigabyte per second on your network read and write yeah which would be great and the so there's four bay is in the back which we can get a separate camera shot for but these are two of these are popular with power is it just poll do you push the tab and then okay gotcha and then pull the handle yep okay so two of these are power supplies and is this a it is standard at all connector no okay well its standard for the enclosure so this is technically an LSI rebadged controller so I think the eggs are attacked and some other companies have the same power supply okay this is 100 volts 8 3 amps output is less than 500 80 watts okay this is a fairly beefy 580 watt power supply yeah and you've got two of these for redundancy I guess and then it can go up to four if you end up using drives that are you're saying like 15,000 rpm drives or 2,000 rpm drives my my start drawing my power so that's the this will be part all part of the server solution that we're going for here basically we it I think this part of it the CS 381 really is not necessary in the strictest sense of being necessary but I really wanted to work with the chassis and like we were saying an email there's a lot of appeal to having a kind of one box that does everything yeah even though who was your guy who made the cat or was his name the canon cat Oh Jeff rescue even though Jef Raskin you might say that one thing that does everything's not a good idea I like that too for like a home server this kind of box is perfect I mean my home server currently as a fractal node 3 or 4 with six drives and so like this would be the next logical upgrade if I wanted more drives because those those bays are not hot swap base either and you've got eight drives here so I mean you put 8 10 15 20 terabyte drives in there it's more space than most of you actually need at home right I don't like Reddit such are such data hoarding is probably freaking out it's fine yeah yeah yeah they're freaking out only a hundred terabytes me crazy this is this is what we use for the level 1 storage server so we have 170 2 terabytes across some a couple of shelves okay and that's worked really well for us from like since it's like 2017 and and you have an old video that's is that where you repurpose the Google server is that a difference yep same project okay so our Google server is your silverstone right right yeah so what I want to do with this is a storage solution and then also run some stuff on it so we have two main scripts that we run currently I do them manually but we could set up with I don't know if it's would be a cron job through a cron job fast or something or even a full separate virtual machine you can actually run Windows as a VM to do that if that's how you do it now okay yeah so what we do is we have two scripts they're just PowerShell scripts they're really basic they more or less run one of them runs handbrake the other one does just file deletion and what we do currently is compress all the old b-roll footage so once it's maybe 3045 days old and we're not likely to put it in a lot of videos going forward we compress it down a lot like a whole lot but it still looks pretty much native quality we did a test ages ago and you couldn't really see the difference and that's done currently with handbrake and then we do file deletion for old a role so a role would be like this like the standing shot once it's in a video if we ever need it later we just pull the clip from the rendered video keep that full quality and delete all the a role so that'll save a lot of space and that's how we've stretched out our current mask for as long as we have but the cool thing will be adding additional storage 3-year - ok so let's I guess let's get started on stuff this is gonna be a computer build but we should probably set up on a bench first to make sure stuff is working yeah because this is a technically a small form-factor case before you mount it in there let's make sure all the parts are good is working on it after that not fun yeah motherboard first motherboard photos did another word first so this is that $500 board I mentioned that you recommended so it's there is a version of this board that's like 250 mm most of the costs are almost half the cost is the 10 gig interfaces so we've got two Intel X 550 10 gig interfaces so 20 gigabit plus IPMI plus onboard VGA IPMI we should go over internet protocol management interface so you can into the computer even when it's off so when we got this board we want to use a risin 36 yeah we have a saw it is not a 3,600 X I pulled that yeah 47 so we had an extra 3,600 and an extra 3600 acts so we're gonna pull one of those for this system and the cool thing about a.m. for obviously is that it shows a sock with a lot of things although we're at sort of end of life for am for at this point but we can upgrade to a 3900 X later if we wanted to we had extra 3600 series that'll be good for starters and then I guess the the other thing I was most excited about was just ipmi yeah we can remote into the board and so when we first got it there was no support for third gender isin and so a lot of boards have bars flashback and like you make a USB stick and you got to rename the file and put it in and look at some you know read the tea leaves or the blinking LEDs nah we just went into the web page on this thing and it's like hey I need some new firmware and so we did it the other nice thing is this board generally has more ECC memory testing so unlike like with the Intel Xeon x' you got to get well you get a Xeon if you want to err correcting from Intel but with Rison as long as the motherboard vendor has tested it you're gonna get ECC on just about every cpu one other now I mentioned on board VGA and the nice thing but the on-board VGA is that it does not consume your PCI Express Lanes so we have to buy eight PCI Express Lanes and one PCI Express 2.0 through the chipset we should show that to so adding the lanes up as we did previously it's so we've got by eight by eight by one mm-hmm what does this consume by 4 by 4 and then we have MDOT to drives over here one of the m2 is PCI Express 2.0 by 4 3 no chipset the other one is PCI Express 3.0 by 2 so we've got two extra PCI Express 3.0 one it's right they were repurposed from SATA Express yes so this actually does make some some use of that and the onboard video is cool just because Rison clearly unless it's an APU does not have video capabilities but for a server it's awesome yes because we don't I mean we'll probably put a decent video card in here maybe for compression if we can get it working yeah like in 5700 XT or something but also we don't have to so that's always nice this Ram is for people watching this is just extra 4000 megahertz memory it's not not not something we're gonna be using for future testing so 4000 mega Hertz with rise and just doesn't really this specific kit it's got bad timings it's running outside of the Infinity fabric one-to-one and we're just not going to use it so we'll throw it on the server there you go our GP server so you get this this is waiting for BMC to initialize the BMC this thing is it computer it's a computer within a computer so if it's got its own RAM and its own little controller and that's what we were using to update the BIOS and check on the system over the network so now we're just booting a test bench on the other side of the room to get a web browser and this is the interface that we've been saying nice things about and we'll get some screen capture this to what we could all the voltage that's really cool I was pretty excited when I saw this we have some temperature readouts which is nice CP is reading 52 bit warm but also we have our PM readout somewhere here you go so fan one is reading out 1200 rpm that's gonna be the CPU fan BIOS firmware version I think and pretty much everything else except EMC version was all blank and we updated through the interface so the process for that was to go to the maintenance tab so this is all accessing that motherboard and then do a firmware update for for a firmware update and it's got things like preservation of different settings if you want to do that and then there's also a BIOS update I want to say that I'm impressed at the back button works on this but because that normally doesn't work in web interfaces for network it's a start where it only bugs out preserve BIOS configuration I thought was really cool too for for attempting a flash we can launch a remote control for the BIOS we can control remotely which is really cool I'm happy about it because one thing is with travel a lot of travel it does mean that if something goes wrong I can more easily try and fix it remotely and then the other thing too is we technically don't need a monitor for that once it's set up we can just use a remote interface and we mentioned adding a graphics card but the magic here is because the the VGA is built into the motherboard so it with an add-in graphics card you would not be able to use not really not generally not be able to use this type of remote control with an adding graphics card hmm there we go so there's all the sensors that are reading out to that interface page 2 and they're reading the same thing so if for some reason we were worried about accuracy so this is all reading and I guess we were set up check out event logs ok logs and reports logs and reports ipmi yep so ipmi event log there is a limited amount of space that is actually on the motherboard that's away from everything else and it will record stuff so you can see when we first powered the system on and it didn't really know what was going on is like you know time stamp ok so a clock sync happened and it's like this is a lower non-critical something has happened this is oh this is when I stop the fan yeah when I stop the fan and the LED came on there's a log of that mm-hmm in the system that's cool yeah so any kind of anomalous event will get logged here let's let's do a demo that let's just unplug the CPU fan will it if yeah I was gonna say if you stopped by hand well it's still yep alert yeah so I don't know if you can see the there's a tiny blinking red LED there right next to the connector and so the connector actually blinks to let you know which one is being problematic so if you've got like the zip-tied bundle of fans and you don't know which ones which there's an LED yeah which is nice and I think if we refresh this page a video log is really amazing also we'll come to will come back to that okay so there's your same is that the same yeah yeah so Santa named fan wants him to type fan and it just says going going wow is that the RPM yeah yeah it was asserted and then D asserted so it the asserted means that it's no longer asserting in the error video log is really awesome so once we get an operating system on here let's say Windows blue screens this thing can detect the windows has blue screened and it will playback a video of what the computer was doing oh cool a second or two before it blue screen that's extremely that should be on every computer yeah yeah yeah that's like how much does this cost can I get this on the next like high-end gaming motor yes that would be great I mean I'll take it as an add-in card it's fine yeah that's extremely you are they're adding cards or stuff like that there used to be but I haven't seen any in a very long time okay okay so I guess we'll shut down and start putting it in a box so you might be tempted to use the 8 SATA ports on your motherboard but in general I don't recommend it and the reason for that is I've generally not had a good experience every time I've tempted I've been tempted to do it over a long enough time line like for me I want to build something like this and stick it in the closet and forget about it right I want to mess with it ever again and so I always recommend some kind of an add-in card for managing your connections to physical devices and because we've got the external disk shelf something like this LSI controller which will support nvme but also serial Attached scuzzy and it will connect right up I mean it's the same connectors as the drive base that are built into the silverstone and so this will support 16 physical links will have eight physical links going to the drive shelf and eight physical links to the drives physically inside the computer but look how much bandwidth we're wasting or we're gonna put eight mechanical hard drives in here which would be hard pressed to saturate even one of these connectors whereas we've got 24 drives in the disk shelf that are gonna use to physical connectors which is a much better use of bandwidth it's also true that it's a redundant connection so there's two independent back planes and two independent controllers and each disk has two independent paths to the controller so one channel on the controller could be lost and the whole system will not hiccup it will not go offline can I point out how thick this paste is so this has been on here since we got it and it's so it's such everything this paste well yeah all the liquids evaporated it's actually ripped part of the shop towel off that's definitely needs to be replaced that strongly suggests that uh these kind of coolers that have been sitting on the shelf for like a year should be pasted yes all right so we're getting started here I'm just gonna sock it in the cooler and you're working on the power supply so it should go pretty smoothly now that we know the motherboard works I'll do a quick plug for the store if you want to pick up the toolkit I'm using or the mod mat were working on you can get Astro dock here with excess net the mod mats are back in stock now in shipping and while we're while we're shilling for things I'll point out that level one text you should subscribe to level one tax will link them below you've got a store as well thank you it's fine is it store that store Duck global what X like to have its own formulaic did you ever see that that I don't blame you if you did it but we bought a case in China the three-body does that sound familiar to you mm-hmm my reason I'm thinking of it so this is we couldn't get that to happen on Cameron yeah so I can definitely be me well I think we're gonna have another problem with that when I get a little farther in it might clear if we pop the plastic thing off yeah yeah you know the pop the plastic off got a DRG be the CPU cooler you want to try and fit it first like that yeah before I get destructive we can maybe destroy it technically yeah technical year that's a little rough the fan spins once you put go and I wait wait cuz that's got a push down right yeah so it's so now set well I mean you could put the top on probably wow that's really not good cooler did not did not pay the least bit of attention to the keep out though well like not only is the cold plate a lot smaller than the IHS it is it's not possible to install it like if we put one stick around like that's our option okay well I hated this coil so it's perfect really you can manage a measly little 36 is it's fine I just never liked 120s because there's such bad value versus air coolers oh yeah but also for this odd use kids know even for breakfast no I haz am for but it might be an early sample if it's an early sign for it oh nice okay cool so I do have one that fits okay oh wait sorry needs just Ram we can use all four RAM slots yeah it's amazing isn't it just barely though the cables right in their meat know what they can't get it high enough on that side cuz like bracket what's it hitting Oh everything was not to be this case could be so much better if Silverstone had like increased some of the clearances by about a millimeter just sock it down yes holding my breath for that it was because it was a clipping right there at the edge is that panel gonna close probably Hey alright or do we have a hemmed up to the mountain here because that's gonna be inaccessible so implying that for the OS it's it's the main like storage array the OS technically runs from a USB stick Wow okay oh so that's done alright I think the last ingredient is this side side panel whatever whatever we're going to do to get some airflow over that because this this is a Dante's Inferno in terms of heat production I don't know if this one actually work I don't think so no and then just a mask off and we bought these in SE GE market in China and that motor is crazy you should feel that feel the wobble 0.85 amps I don't know if the headers are rated for that they might might be able to do one moment of truth it's like can we wedge this in as well it's tight so leave a second one out I guess we can rebuild it okay we've got our drives installed everything is magical now I needed to move on to soften we did rebuild it though yeah we've we rebuilt it because I think we we found that cryo rig cooler we had an extra one I don't know what it's called but maybe the c7 but we got that to fit on there it blocks the first RAM slot so if you're gonna use that cooler and this just be aware yeah well if you're gonna use yeah this is not a bill that would precisely replicate in general but so we rebuilt it I feel a lot better about it now the cooler smaller it's probably not quite as good as a CLC but everything fits really nicely and we got both of the drive bays in there so we're good with that I guess now it is time to boot with on raid so we've made a nun raid key you can do oh I should also mention we flipped the the vrm heatsink around oh yeah that was necessary to make the cooler clear but it doesn't it won't affect I'm a p-problem want to pick UEFI mmm the reason you want UEFI is because we're eventually going to build a VM with a graphics card should we change anything in BIOS first or is everything know there's stuff we probably wanted to change about us but to close out this video we can just do the UEFI corsair voyager blah blah that's not good oh wait yeah okay under it does this really dumb thing where you have to boot into the GUI to be like no I really want UEFI and then it's like okay so there's a yeah I forgot there's a software I pulled that that took me 30 minutes because it was like what why on earth would the thing be set up to not work in UEFI 2019 yeah and it turns out that's the thing like you actually there's a there's a GUI setting in unwrite that's like yes I really know what I'm doing I want to boot in UEFI mode and then it's like all right there you go so we're booting to the USB device now and I think this is probably a good spot to close we've this has been let's see it's go for this is currently 531 and we officially started maybe 11:30 a.m. so with a break in the middle but still like maybe four four ish hours of real work to get here and every pastor syndrome it's really triggered that's super trigger because it's like how many people does it take to build a mini ITX computer yeah but really not even many icx but to be fair I think also when when you're pretty familiar with the process you get a bit perfectionist about stuff like I don't really want a cooler that kind of works in there you know so of course we want to unbuild it and rebuild it yeah whereas I think if you don't know as much you're probably happier with just just make it go Jordy and it's it's so quiet and clean now yeah so much better much better oh yes we've also named the USS excelsior because I have named all of our NASA's after after the Star Trek ships and this one you have to cut in the noise that it makes when it tries to go to work - fire fire fire birthday in spirit for I think and that's because it's kind of cobbled together it will probably resemble that ship so for the Star Trek nerds there you go but yeah we'll close it out here part two should be the last part not trying to make a lot of videos out of this but we have several hours of content we're going to compress into 30 minutes from this alone we might do a separate upload with bonuses shots if there's anything interesting and the unread content is gonna be unlike any other unread content out there because it's ZFS on on raid that's not a thing yeah so check back for part two subscribe to level one text link will be down below and I go watch this on his videos you go to store documents access dotnet to support us directly or patreon.com slash gamers nexus and check back shortly for part two we'll see you in the next one [Music]

Channel: Gamers Nexus

Views: 689,520

Keywords: gamersnexus, gamers nexus, computer hardware, level1techs, wendell, diy server, diy nas, diy nas build, diy nas build guide, diy raid, unraid, unraid tutorial, zfs tutorial, unraid zfs, unraid with zfs, set up unraid, ryzen server build, diy home server, home media server

Id: Hix0l8cFaMw

Length: 35min 8sec (2108 seconds)

Published: Thu Oct 31 2019

Reddit Comments

[u/eppic123]

When gaming hardware YouTuber dip into server/enterprise stuff, it's always a wild ride.

[u/BlanchDolor]

Loved the intro. Sub got a mention too!

[u/Reset_Assured]

Im surprised nobody has used proxmox with ZFS + ceph since it has an OK gui for that.

[u/xenago]

I don't really understand this logic. Why use unraid if you don't use the unraid features? Just install debian or something...

[u/ktnr74]

I am more concerned about how this video is going to affect prices for the DS4243/6. I was just going to pick up a couple more.

[u/AshleyUncia]

Hardware wise, I just don't get why they didn't have the cooling planned out. A lot of those weird 'consumer server, but tiny' cases and such have pretty weird physical limitations for different components. My first instinct would be 'So with this case and this MOBO, what coolers can we ACTUALLY fit in here?' They kinda just tried to figure it out as they went along.

[u/BloodyIron]

ZFS on UnRAID

twitch

[u/ktnr74]

When gaming hardware Youtubers make videos about gaming hardware - they almost never forget the ROI point of view. Like which specific video card is the best bang for the buck at this specific moment.

When they buy/build systems outside of their area of expertise - oh I liked how that SilverStone CS381 looked at CES so I am going to get it - who cares about the price?!