The Homelab Show: Episode 2 NAS & Storage

live now it's like all right tom here from orange systems and jay here from learn linux tv yes we are here with home lab show good news a little housekeeping will take out of the way we did register a domain uh but through a calamity of errors we own the domain the homelab.show it doesn't show anything right now so um i was hoping my goals my lofty goals were to have the podcast ready but yes we are converting this to a podcast yes though all these episodes starting with zero till current ones until it's live will be published and available as a podcast download um it may take me another week to get well in less than a week but it just didn't get done today because of a hosting issue that came up but nonetheless we did buy the domain so the progress has been made and yes this will be a podcast so um we'll get that out of the way there's our housekeeping for the day but it's happening that's the important thing it's happening yes uh we've been listening to the feedback we'll continue to listen to the feedback uh but right now we got a show to do and it's talking about where to store all this because i mean even these recordings have to go somewhere so this is the often debated which one do i buy um or build or uh file systems and everything so let's talk storage yeah it's the one thing that unifies almost every home lab if every home lab depending on context because you have to store something somewhere and how many home labs have started because i want to run flex where do i put all my movies i mean that's not every situation obviously some people are running virtualization solutions and whatnot and they want you know shared storage but you know we have to put our stuff somewhere even if it's just family photos or something yeah and the debates that go on um i will admit and and i'm okay i will accept this from the very beginning i'm a big fan of zfs and it's okay if you say tom you could be in a cult i'm there's a chance i mean i am in the cult of zfs this comment seems to be randomly scattered around it because i've talked about zfs too much on my channel um also wendell from level one tech's definitely in the same cult as me he's a huge fan of cfs as well yeah i don't know if i would call it a call because i think they're it's hard to tell the difference between someone that has chosen a solution that fits their use case specifically and if something works really well for you you're probably going to speak very highly of it because it's human nature so it's hard to tell the difference between someone who's legitimately into it and enjoys it and works for them or a cult right because in both situations you are very excited about it so um i think it's fine personally i'm a zfs user as well i don't think i'm you know as high level as you are um on that but it's what i use i use journals as well but i think a lot of this is just kind of talking about what are the options that are out there i think the reason why there's a debate is because one in my opinion is not astronomically better than the other if you look at pros and cons you'll find possibly that they're even in their differences it's just a matter of what is important to you and your use case and we could probably help you understand some of the differences between them in this episode yeah and you know i figured the file system's kind of a way to talk about uh things from the beginning we'll get to the nas and some ideas around nas secondary but you know the nas does have to run a file system so that's a good place to start now the cool thing about zfs and you know i'll say my piece on this is zfs isn't just a file system it is the integration of the file system along with the block level devices themselves the hard drives where the data is stored that makes zfs a little bit different than some of these systems like the mda md adm raid systems i say that right mdadm all right raid systems that are in linux because they're managing the hard drive but then you can format it two different file systems on top of the layers so you have two different components managing it um and this can be a little bit confusing because they're like well zfs just kind of does it all versus not doing it all but there's some advantages when you go up at scale um and we've had people that set up servers that are you know one of the ones the reviews i'm working on right now is a 420 terabyte storage server and it gets even bigger from there for some of the clients that we've done even larger servers for um you become really interested in how all that data sits because you got worries like bit rot you got worried like a lot of other functions and yeah so from a high scalability zfs seems to be a good solution now swing it back the other way um in synology so we're talking about me and j are both familiar with them and they use a kind of an interesting mix because you use the md adm raid system along with butter fs and ext4 they blend all of that together um because butter fs was a is a competing file system for those btrfs often i've always heard it's called butterfs but butterfs is a system that's made to try to compete with zfs like an alternative option that also supposedly will do not just file system but integrate the drives but this is where the split comes in for stability there's some stability issues using butter fs to manage the drives this is where synology made a hybrid design of using md adm your linux raid versus the um you know using butt rfs to it they still get the advantages the snapshots and the journaling functions that you get with zfs or butter fs when you use butterfest that has those options but does not they do not use it to handle the drives so i think we should probably talk a little bit more about that and break some of it down obviously we've got to try not to take up the whole episode to talk about file systems but um i almost feel like cfs should be considered something else than a file system yes it contains a file system but it's so much more than that now when i first started i mean we're talking you know windows nt4 and you know red hat desktop before red hat enterprise ext2 file systems back then if you were to ask me what a file system is i would say well it would in my lingo back then i would say well it's a it's just a table on a hard drive where it's indexing where things are i would have gave you like a simplistic answer that's not completely true um but that was my mindset it's just the way data is written to the disk which is actually what it is but when you talk about cfs you have tooling on there and an intelligence that's built in and you talked about um you know death scrubbing like integrity checking if a bit gets flipped but a file system and this is confusing because i could say the linux file system i'm talking about the directory structure but file system as far as the disk is concerned it seems like the whole idea of what a file system is is changing a lot they're much more intelligent now and zfs adds a bunch of features on it and it was basically made famous by bsd and it's still obviously there there's some controversy i won't get into about it coming to linux in the form of open cfs which is largely shouldn't be a problem now so zfs is a contender butter fs which i've heard some people call it btreefs um i've heard that before and i was confused and i uh i've heard it called butterfest from people who work at ubuntu so i assume they're right it was that it was at the penguin conference you know i've heard people say zfs you know there's different we've been calling for everything yeah if you're over in europe it's zfs is that a fest because they say z is zed or in england so so understanding the context we're talking about how the drives are formatted and that is one of the important differentiators between the solutions because the underlying file system as in the intelligence around the storage and the hard drive it's something to keep in mind it could be as simple or as complex as you want maybe you could just have a simple ext4 disk on a on an ubuntu server and you just manually set up nfs it's a file server i mean it's a nas it's not a commercial one you built it yourself it's perfectly valid you can roll your own and that's an option for some people and some people love that because they get full control other people want something out of the box something that they could just buy plug in um just set up through the gui through the web console um and that's valid as well uh maybe they like synology in my opinion it's better out of the box and we were having discussions offline that it's really great for enterprise users through nas um you get additional features in a different direction they're pros and cons so i think we could probably walk through the different solutions and then what sets them apart and that might be a good basis yeah before we get into that i will mention because i seen someone mention it's in there and this is still separate from because i i think the homeland people are more going to be in tune in this world will keep the conversation going um with building something like a true nas unraid or using synology of note you can buy enterprise level storage equipment that does have its own raid controller those i i don't think we're gonna spend any time talking about them they exist dell makes a lot of them they have the pert controllers they do have a use case they're not bad um you don't get any of the fine green controls you're going to get with and where i see more of the home lab people have an interest like you do with uh like a true nas system but they are valid they're good uh they're they're more what you buy is what you get there's a lot less tweaking you do with one of those to give you concept to how they work though when you buy something that's like an enterprise rate controller you'll have 10 drives in there controlled by the dell per controller but as far as the operating system is concerned it sees just one hard drive that's less interesting and usually not where the it's not where you get the most performance on there and tuning those is well you're dealing with a proprietary controller that they had and it's uh i don't know i'm less than thrilled with those what we are going to more focus on is the is the nas solutions and if you're using an enterprise storage server you need it to be in whether you call it it mode or passenger mode we're talking about the operating system interacting directly with every individual drive um and i know we mentioned to see a little bit where talk about hardware last time but yes there's ways to re-flash firmware someone said it's easy just follow some tutorials it's better if you can buy them without it being flash so leave it at that you yeah in just just for a bit of context and and i think we have mentioned this before you can go on ebay and buy it in whichever mode you prefer so if you want the operating system to see that there's a raid controller or not um depending on how you want that set up and then the term jbod comes up a lot just just a bunch of disks i believe it stands for yep and um that's important when you're considering trueness because you want trueness to control the disk you don't want something else to control it zfs wants to be in control of it and that's one of the one of the places where this comes in mind and where you have to pay attention because if you do want zfs if trueness is your decision then you probably will need to pay attention to this if you're buying a server on ebay or something yeah it's something to look at and the good news isn't like i mentioned last time you you type in the word free nas or even like freeness super micro in ebay you'll find a bunch of them already configured in the mode to allow that so you don't have to do as much tinkering um you know that's handy that because there's enough tinkering you'll do trying to figure out because the next question is how do you set all the drives up or what's the most optimal way and boy that's a tough question to answer but let's uh it is yeah i mean one way we could think about it is several scenarios a you have a desktop lying around b you want to buy something like a commercially supported device or you want to buy a server that can help differentiate the scenarios a bit and um because you could just install prunes on the device buy something internet's built in bias analogy qnap robo whatever you want and that's really up to your use case yeah well let's talk about a couple of the nas ones out there now i are you an unraid user at all as i'm not um i am not and i feel like there's some of these things i'm not going to be able to talk about as intelligently as others and um that is actually one i would not be able to speak so certainly about i've dealt a little bit with it just you know from client interactions and so i'll mention i have no problem with unraid if you really like unraid there's a couple of really good channels that have more in-depth tutorials uh on that it's not hard to do do a quick youtube search for it i think unrage really popular it was made linus tech tips probably really threw it up there in the popular world because it's a great solution um for a lot of use cases the things i don't like about on raid as much is it does not to my knowledge natively right now there's add-ons i know uh support zfs but the advantage you get is the fact that without using zfs it has more expandability to just add more drives to the unit so i have no problem if you want to use the solution i don't i have no reason to tell you not to use it so to speak if it's the preference you have if it scratches your itch awesome what me and jay definitely know a lot more about is the two other ones we want to talk about that's going to be truenass which is formerly freenas but truenas and synology yes that's going to be um a big component too so and also you know i do want to add out there like manually setting up your own ubuntu server i mean we're not going to walk you through that in here obviously but that's an option too for more manual approach but i recently had a chance to check out synology and just for um completeness i am a true nas user i have been for a while before that i was running a manual linux server where i just hand coded all the nfs configs ip tables samba all of it by hand and i moved from that to synology and then moved from synology to trunas and and by moving to trinas from synology i'm not saying that synology is inferior there's a few quirks that we'll probably talk about today about synology in the context of a linux user because i think that's also an important thing to keep in mind too there's some quirks yeah synology does make and i get it depends on how deep you want to go from an out of the box configuration synology is relatively easy to use their plug-in nature they're kind of i like their ui it's very intuitive i would say it's very um it's in a browser but not like a typical browser menu click like it's got draggable windows for those of you that haven't seen it or watched any of the reviews has your review been published yet jay it has not it's been recorded i'm a bit behind i still think it might land this week i have a video where i'm going over open media vault another option in fact this is it right here um this will be in the i'm editing it right now in the queue i'm going to talk about how to build your own um open media vault server with a raspberry pi in a m2 storage that's this week synology this week as well so i guess on my channel it's now speak pretty much i yeah i have a few videos on it but their their interface leads it to be relatively easy the other thing that's really you know easy for users to get started with when it comes to synology is going to be the fact that they offer easy expansion of i only have the money to buy three disks now and you know uh constraints monetary constraints are really a big part of home labbing because you're often not in the later parts of your tech career you're in the i want to get started in tech that means i don't have that money just to run out and buy 10 16 terabyte drives and build it right um synology does do a nice job of letting you buy the basics and then grow into it that's definitely an advantage if that's like one of the check boxes that you're really looking for um i would definitely go say yes on synology also they're kind of almost like a one-click install their application store is extensive there's a lot of things built into synology that are make it pretty easy to get going right out of the box like this you run through a little application let's click this oh i need to i want to back up my pc um my windows boxes for my family cool they have a complete native backup computer system built right in that stores it right back to this analogy that even has a whole bare metal restore there's a lot of that functionality that even for home users we've seen is awesome especially if you're playing with home labs you want to be able to tear down machines and rebuild them quickly synology has that way on there you want to back up the kids pc because they're going to click on something and you would love to be able to restore it right back and it's not that you can't do that and use true nas as a destination it's just like i don't have you used backup pc and some of the ones integrated into nas they're not as intuitive i i have not but i did get a chance and this is going to be in the reviews i'm giving it away but um the docker integration is fantastic in synology like i think my complaint is that it's too easy like it is so good like you can download the image you can run a container from that image you can have it always running there's a check box for it and since synology is a storage device adding storage to the container is easy to do so you could basically create a shared folder in synology you can mount it to the container so the container is able to write as configs to something and then you can view the logs for each individual container in the same menu and the reason why i say it's too easy is because if you are studying docker it's going to completely abstract how it works underneath the hood but maybe that's okay because maybe you want to run containers and you don't really care how it works behind the scenes but if you do care you could uh explore that further on another machine but their docker support is really good i was actually very surprised by that the way it just pulls right from the docker repository you know i sing a lot of praises for synology because it really like jay said do you want to know how it works is this what your home lab goal is or is your home lab goal i just want to run plex and be in control of my media i wanted to not have that reliance on third-party services but i'm also not a technical person who wants to know exactly how docker maps storage or how docker maps networking because they do they make an interface on top of it that like jason it's a few clicks and next thing you know it's just working um you know you can run it run through a wizard next time yes your way into having a docker image and i don't see that's necessarily bad i think it puts technology in the hands of people at a faster rate so the overall on that is still a pro on there i think it's true that synology has for a beginner home lab user the most ability to potentially become your only thing if there's someone out there that wants only one device they want a device let's say they want to run plex and they want to have storage for the movies i don't know how well plex runs on the synology so i can't personally speak to that but you probably can do it but speaking in terms of docker i was looking at the container i was running and the unit had four gigs of memory ecc memory and the container was using 50 megabytes of ram now if i was to set up a virtual machine and try to run that same app it's probably going to waste 512 megabytes to a gig and that's where we get a little bit of trouble because it it ships with four gigs of ram yeah and that'll go very far if you want to run nothing but containers because like i mentioned my container that i was running 50 megabytes but then if you're wanting to run a bunch of vms okay now we're starting a little get a little constrained here so but you could expand the memory in these units i think the one i reviewed can go up to 32 gigs is 32 gigs enough well i don't know are you running java apps um so if you want to have a a one machine that's everything if you're running a bunch of containers you want your storage that's perfectly fine for you but if you want like serious virtualization you might need a separate server but it is possible to run virtual machines on that unit just be mindful you can't run like 15 virtual machines at the same time on its analogy but if you only need to run one or two it might just be fine for you yeah and someone said synology is a little pricey for the hardware you get that's kind of a mixed bag because you're not really i mean if you tried to spec it out what processor they use which are usually the lower end not really high powered ones so yeah if you're going to build something we'll get to that in a second with the truenast um you can for that dollars build something that's got a little more horsepower a little more memory but to the other side of that um it's x penology i haven't used it before i know people ask about that which is the running it on yourself it's not the same and doesn't to my knowledge have the full support uh or work as well as a native synology does you're paying not really for the hardware you're paying for the way they fund the development of all that ease of use that they baked into the system um that's kind of what you're paying for it's not as much the hardware it's the hardware and the profit goes into the not the lack of licenses required i should say in order to do it because it's not like they're going hey we do have docker but that's a five dollar a month fee or anything like that they're not trying to nickel and dime you not saying there's no fees because you can also use it for example for a camera system and i've done we've actually deployed a lot for clients technologies surveillance station which has perpetual licenses that you buy so you do have to buy licenses for the cameras but they're one-time purchase and they are transferable to another synology so um they do have a couple things like that like some of the advanced features of virtualization are licensed but the basics aren't in docker's not and all the backups features that it has like office 365 backups email backup solutions matter of fact they have a whole mail server system in there there's a couple things that uh they do also include in there that are kind of interesting i think it's everything but the kitchen sync is what's in synology but i actually speaking of price i compared it to the trueness mini the base model so the unit that i re i reviewed that's going to be on my channel i think it retails at 7.99 for the synology that i reviewed and again has four gigs of memory so the base model trueness mini has eight gigs of ram so it's four more gigs of ram than this analogy at the same price however you also lose two drive bays at the same time so yeah way what's more important memory or having those extra drive bays and with trunads of course you could build your own or you could just buy it to an s mini if you'd like um there's a few options but if you have something lying around like a spare server a spare desktop if it's up to the task you could actually just install kunas on that device yeah now the the last comment i'll probably make on synology is the the um wattage is probably kind of another one more to the synology side when you buy a complete device like that you take the guesswork out of building now some people are going you're taking the fun out of building because i like building things and tinkering with it um but there is another class of people that go i'm here to learn software development i don't have time to learn why uh random reboots are happening there is a little bit of advantage of buying a complete you know this software works with this device low wattage their fans are generally quiet with their desktop units um specifically they're really low noise so you kind of get that and someone even commented like their synology idol is like using 20 or 30 watts like yeah that's normal that's they're yeah so there's definitely that's kind of the last thing i'll have to really compliment on synology and it is true and to be fair you can achieve that similar thing with trueness but you do have to put a little bit more work into it the one behind me which is in my rack behind me the blue one is a true nas server takes about 50 watts of power somewhere between 50 and 60 watts of power for the whole thing and i achieved that because i put a lot of research into the parts it wasn't like a oh i'm going to buy everything right now in 15 minutes i took some time like i don't know if it's a week or two and also i have the time because i'm going to make a video about it so it's in within my best interest to really make this work but someone might want something turnkey so now it is great for that but you can achieve that with trueness but like you said tom you have to really pay attention to what you're buying and compatibility in that to make that work more yeah so now let's talk about the fun part let's build a true nas and this can um cross over to on raid as well the hardware that you choose for this whether you want to build it on an old server or you want to build the hardware yourself you know either way there's definitely options there but it comes down to picking the hardware this is uh on the trueness forums i i do like they have a whole dedicated section of the forums called will it i think call it was called will it free nas i think it's called will it true nast now they've been changing names um but it was kind of where you could ask the questions of what cards to use and things like that and i'll give a big shout out to serve the home they they've been doing a pretty good job of keeping this up to date they kind of keep a list of recommended popular hardware to build if you're going to buy parts and of course every year i i think it's brian moses who does this he does this uh really nice blog post where he details out hardware choices for truenast uh it gets retweeted by truenas you can you type in like um what actually he's almost always the top result because he's been doing it for so long we can leave a link later in the descriptions but either way it's there's a lot of different guides out there for building your own you can even if you don't want to buy something from ix systems and you you know they do sell like their true nas minis which are turnkey complete so there are complete solutions you don't have to build everything your own and if you go to the enterprise level ix systems offers and um you know really high-end custom-made servers and we've we're a reseller so we've actually sold some of that equipment as well but if you want you can just look up the specs and the hardware used in like the tunas mini and yes it is loosely based on well not loosely it's a very slightly modified like super micro system so yeah you can go ahead and go there get that type of system and buy those same motherboards buy those same parts i think super micro makes a case that's almost it will they oem it for ix systems or something like that because it's about the same case but it doesn't have quite the same branding but it's still mini a nice small case with a bunch of removable bays or you can go look at some of the um the different hot trays if you need the hot trays and you want to do absolutely custom build it yourself yep and normally what i would tell people is since i i have a true nas server that i built you can watch my video you can look at the list of parts that i use to achieve this but the reason why i'm not telling you to look at that and not telling you to look at the parts list is because i built that server like right before the pandemic started and now the prices of those parts have skyrocketed so now you look at the you can look at it um now that i've given you i'm giving you the disclaimer but it's like it costs what that motherboard costs what i could i could buy a whole system for that price um but um it's not what it cost when i did the video which is really complicating things right now some of these parts if you want to buy them new is kind of hard to do actually for a reasonable price yes definitely um some some challenges there for sure but if you have something lying around and i keep mentioning that because i think that's how a lot of home lab people start because they have this old computer maybe their previous model or another thing they were using use it try it right it's just collecting dust anyway and sometimes that's how people start because they don't always have the money to go buy something new if you have something you could try it out if you have extra hard drives i mean that could be something where on raid would shine if you have like uh you know i have a 500 gigabyte drive here i have a one terabyte here and a two terabyte here or whatever you have they're not all the same they're all different ones you know 5400 rpm the other 7200 so even the speed is different and i'm pretty sure isn't it true that on rate is is what you would want to use in a situation where you have unlike drives yeah um unraid because it has a different um it's not using zfs and we'll get to that in a second exactly why you need to build zfs the way you do there are ways you can mix and match drives more so in unraid just like synology on raid also has that as kind of an advantage of being expandable so to speak because they're basing it differently they're not doing zfs once you come over to zfs and this is going to be the same for on rate if they if they decide that zfs is the way is i believe you can use zfs as an add-on from rate it's just not native right now like i'm not an unrate expert but if they decide to go to zfs this will be a challenge that that comes with it like unread will be able to say oh yeah you can mix and match and do different drives and expand the array oh wait unless you build it based on zfs now you wouldn't be able to um you're going to do that because the limitations are not true nas the limitation is the way and the nature of zfs there is a way to expand zfs uh but you have to expand it with symmetry um i did drop in uh the show notes it's it is brian c moses.com he's got a a diy nas uh econo 2020 and diy nas 2020 edition he's been doing these for years like they go back for years and years with lots of details on hardware and everything else so it's a great it's a great reading breaks down all the little components and talks through a lot of why he built it the way he does and there's one aspect of zfs and i do want to cover i don't know if this is the best place to insert it but i think it's an important thing to keep in mind um because you know when you make the mention like what's required to get into it what do you have to have to start running cfs then some people might think wait a minute if it's like i have to consider all these different things just to get started with it then that's kind of annoying and tedious and kind of overwhelming for me to have to worry about all these different variables why would i do that i could give you an answer that happened to me um i have a ton of pictures probably 20 or 30 000 pictures at least and i think i'm being a little um on the low end on there and one time i don't know if the kids are asking me about an event or something so i started clicking through some pictures and then this one folder i had like 30 pictures and two of them were distorted completely just horrible you couldn't even look at them because they were just completely corrupted now the thing is when did they when did those two pictures become corrupted i don't look at the these my photos every day i might look at them once a year maybe once every two or three it could have been that these files were corrupted a long time ago i didn't know i went to go look at these pictures and i can't view a few of these so what did i do well i did what every new home home lever does i over complicated the heck out of it a friend and i wrote a perl script and what this perl script does is it does an md5 on every single file on the file system and it writes it to a matching text file and if the md5 is different the next time it runs let me tell you it takes days for this to run it'll send me an email say uh yeah the md5 is different on this file of course that is the hardest way to do it but i was very proud of this perl script it was so cool but then looking at zfs it has the ability to find this kind of thing and uh help with bitrot when things are sitting idle on your file system and when you have pictures and memories what do you do you can't view every single photo every single day to make sure that it hasn't you know had an issue you need something that's going to help you check the integrity so in that case cfs however complicated it may or may not be might actually be worth considering yeah and this is one of those use cases so you know even for me personally my data collection days go back to the 90s i still have bringing forward i got if you can remember a sony mavica camera with a floppy disk in it yes i had one of those in 1990 something late 90s and yes i still have all the photos i ever took and stored them now back then i didn't store them on zfs but now i do i've been carrying forward data for years i don't really look at that data too often but it's kind of cool the fact that i can and once in a while i'll feel maybe like going down memory lane and looking at old sony mavica pictures but what happens is the bit rot and we've dealt with this with our commercial clients especially in the film industries they produce very very large files lots of content lots of assets as they always refer to them that asset block they create and we actually have a client that produces high-end commercials like that you've seen on television or wherever the commercials are displayed these days they produce these and sometimes they want to modify or re-spin a commercial but not spend the time reshooting it and it may be a year or two later before a said company wants that asset re-accessed um this is what been the case what's gotten some of them to switch to things that are zfs based because they're going yeah we bought this proprietary fifty thousand hundred thousand dollar storage server and yes that's the kind of prices they're talking on these and it was unable to retrieve a fully working functioning asset they had to reshoot things or like the cost of reshooting was expensive or sometimes not as possible and they're supposed to be the stewards of this like the contract isn't just do this commercial and you're done it's hey we're going to ask you after you shot all this footage to produce a series of commercials over time so there's some real risk that comes in there on from a scalability standpoint and like jason checking all those files that is something zfs does a good job with its scrubbing to stop bitrot so while the complexities of zfx make it kind of unattractive for some people the unattractiveness of losing long-term data is very much there now you can expand zfs that's it's not hard as is many people make it sound but you can't really nilly randomly grab drives and expand it um i've got some videos i've done on this about expanding uneven v-devs so when you're building the pool the pool is where all your data lives and then the pool is built on v-dubs v-dubs are groups of drives and just trying to keep it as simple as possible someone's going to be pedantic and say i'm not explaining it enough detail but i'm trying to keep it simplistic here and as long as those v-dubs are kept symmetric so if i have a v-dev of five drives and i have five extra bays perfect later i can expand and add five more drives i can't add four and i can't add six i can add five more drives and if the first dab was raid z1 the next one has to be raid z1 not raid z2 or you have to rebuild all the data move all the data somewhere else regroup the drives pair them back up and bring it all back in if you want to do it differently but they can be expanded this just a little bit trickier when you're doing it um so it takes a little bit more careful planning an example is going to be we we just quoted a company another company storing a bunch of storage uh for movies they're creating or specifically a sporting event and the way that quote works is they know how many drives they need now but they want to buy some of those drives later so we bought a system and we're going to leave half of the bank empty so we can expand it later and expand it without having to do a full reset so it's able to buy a chassis cost is a little bit higher because it has expandability but these are the kind of planning that you need to do when you're building out zfs just a little bit of work out on there now the fun part is me and jay will definitely attest to is the tinkering you can do once you build your true nas system you have really great level command line access you can build iscsi that's going to be at high performance levels you're going to be able to build nfs shares for your virtual machines you're going to be able to use the acl system tied to active directory if you want to go that crazy in your home lab not every home lab does active directory but there's a reason they do because maybe your job entails using active directory so it's a good learning thing yes you can incorporate it the latest true nas has lots of acl options for granular control of permissions and mixing them all together so you can have some iscsi over here you can have some active directory control permissions over here and it's just kind of a there's a lot you can do it's very extensible yep and i think yeah the tweaking thing has been a lot of fun for me i mean the snapshotting capabilities and the data stores like the data stores are great because you could separate logically your storage however you'd like like i have a media data store and inside there i have a videos data store for the movies and i have a um music data store for all my music that's where all my lacuna coil albums go for example i mean i can go on and then i also have a sync thing data store inside there is a documents data store and i have one for projects one's for downloads and i can go on and on and on i even have one for my retropies i have data stores for those and the way that they're they're logically separated is great you can snapshot those and come up with your own snapshot schedule weekly daily hourly however so you could go back to that point and what's really cool is you can you don't have to overwrite the snapshot to the the source you can actually create a new data store off the snapshot that's the file that it was missing or that you accidentally deleted put it where it needs to go and then delete the copy and anyone using it will not even notice that you were even doing maintenance on it yeah the snapshotting in zfs is one of the really powerful things that make it just a wonderful system because you you got sure back it up everyone should be backing up that's not in question at all hopefully because raid is not a backup raid is resilience all right we got that out of the way but the snapshot system is really fluid and it allows you to very simply go through and say all right this is the file system as it existed at this snapshot and this can be automated in the system and then you can always so quickly roll back to it and because everything of the way zfs works is copy on right it's got a really nice way that it just instantly can spin that up only there's like no delay you're not watching it spin for a long time and like jason you can fork a snapshot to be its own thing so you're like i need to see a slice of what the data looked like three weeks ago and you know so i can jump backwards three weeks but leave my current data completely intact get that piece i needed out of there and then destroy that and not have to worry about it or like you said you can even build it as a another share somewhere else yep you can you get a lot of flexibility one one thing i want to mention though is i do understand that butter fs has at least some of the features that we've been talking about yeah i know it has scrubbing but um i'm not as familiar with butter fms yet i am in the process of looking into it more but i know i'm sure in the comments it's probably already happened and someone's mentioned that so i so there is going to be feature carryover in butter fs i'm just not as familiar with it yet but i am aware of that yeah and and it's worth mentioning someone says analogy does have data scrubbing as well yes they do that that is a function that you can turn on on a per um share that you do in synology so it's not that they are devoid of all these functionalities but at scale i really do feel come back to that very first comment of cult zfs zfs i think does it better so uh argue with me in the comments on that one so someone i forgot who it was i'm trying to scroll back and find the individual made this comment but they were saying that something like they've been a data hoarder for a while and have not encountered bit rot for years or if at all and are wondering why we have well the thing about that is you have no it's like um schrodinger's cat right you don't know if you have encountered the problem unless you go looking for the problem you're not going to go listen to every single mp3 or every single picture look at every single photo that you have and if you're able to look at every single photo and mp3 you're not a hoarder because you don't have that much so when you get a lot of stuff you're you may not know until oh yeah that picture is corrupted and when i open it it's just a bunch of random colors in this one area that doesn't look right that's not how i remember this picture and then it dawns on you i i had bit rot hit me so you never really know unless you're like always looking at your things and you just can't there's just not enough time in the in a day or even a week to do that it is a very challenging thing to see the bit rot um and find it and it's it sometimes takes years it's it's that one file and of the assets they were only missing one video out of an entire library of them that were in there so they had all this footage with my client that was doing the commercials they had a video that was missing it turned out not to be the most critical but it was really scary to them because they were like why is this one broken and but the other ones work around it the one before and the one after it works but this one's broken good news is like they're like this is just a shot across the bow they said like they didn't need that file critically um but what if what in the next question is because um the project of putting a team together hey can you guys view we've got um nine thousand videos can you take the time to view all of them tell me if any of them open like no one wants to put the labor into doing that um you know or even this the sports team that we just quoted one for uh just from the last season because of the high end cameras they shoot they have they're like we got 55 terabytes that's where we're starting at and our growth is because we bought new cameras is going to be about uh we think 50 to 80 terabytes per season so we wanted to last five years plan for that though i mean if you're watching a video a short video uh you know something's that's 30 seconds long if you have distortion in that video for a second you might notice it but if you're looking at a movie are you going to notice that distortion what if there's a lot of action and it's like a highway chase scene or something you're not gonna you may not notice but if it's a really huge problem like tom said it doesn't open yeah of course you've got a big problem but if there's just some little distortion in there in a video it may not be as apparent but if you're looking at a photo which is a still i think that's the most obvious one and i had this one issue where you know i'm a retro gaming collector as a lot of people know and i have like 800 games or something like that and i like to have a digital copy of the game so i have a rom i'm not going to make this an emulation discussion but i for years was not able to run super mario all-stars and i'm like well this game has i don't know something is crazy with the rom or something like that i just wrote it off and then one day i'm like i want to redownload this game why not oh my god it works for for years i just assumed it was just hard to emulate that game no there's just something wrong with this copy of the game that i've had for five or six years i just put it down to emulation is hard sometimes but no that's not what it was it was right wrong so but yeah it's definitely a concern it's not going to happen everyone but as you start accumulating massive amount of data you you are playing a statistics game but right now let's get back on topic of storage servers and nas servers um now both synology and freenas truenas do support backups of things including cloud backups this is actually probably something to really consider and i i want to bring it up because that data center uh event that happened at ovh last week um which is still going on this week yeah the big fire um one of the things that i'm working on is some new videos on this and one thing that's really cool is it's not just about backing up what you have sometimes the opposite is true of using these devices to pull all your data back because maybe part of your lab lives in the cloud and you then you know rely on the cloud provider to back it up this is something where you have a lot of flexibility with your true nas systems not just in the jails but they actually have quite a few different cloud providers they support for both the ingress or egress of data they even have a ways to attach it to google drive ways to attach it to one drive uh to be able to you know uh put that on there oh and someone did ask if your rom was bitrot that that's kind of a question we don't know um yeah we don't we it's you don't always know it's almost like the classic my motherboard doesn't work um could it have been um an electric like a static shock i don't know i tried everything else and it just doesn't post maybe i accidentally static shocked it i have no idea like what is the root cause but there's a picture example i gave yeah for sure i've seen that picture a handful of times it was fine all of a sudden it's not and the problem with that is you always have like like cycling backups and snapshots but if you don't know that you have this problem then your snapshots are going to age out and you no longer have a snapshot from whatever time period that picture was not corrupted and then it just ages out of your storage so that's kind of the problem so the the talk more about free uh the free nash true nas now the other flexibility i think that makes true nash really popular is the fact that you don't have any license fees at all attached to it you know i mentioned that with synology one thing about the way synology does things you're registering and downloading their dsm into their device which does talk to their servers to activate and things like that i keep that in the back of my mind a lot because the a change in a company's policies can change the way a device works i you know the not to throw shade at unified but uh hey hey unify you know there's certainly been some controversy where they didn't require registration they decided you know what we require registration now on that device that you bought that didn't require registration they changed the terms of the deal the nice thing about trunast you can just download the iso it does not reach out to some activation server it's not chattering away going hold on register with us to get a license key um there's none of that with truenast that's one of the reasons from a home lab standpoint it's much like a linux distribution is i download it i run it it didn't require me to go out and activate it this is really one of the four core fundamental things that'll make a lot of people running home labs happy is trying to have a little level of autonomy over the things they have they give you that control and you know this is like xcp and g like we talked about last time with hypervisor same thing they're like take it here's all the code it's not reaching out it's not phoning home and activating special accounts it's here here is the xcpng stack and the same thing truenast follows that very much the same you're like download and do what you want with it essentially like block it off on the internet doesn't need access to make it work that really pushes it on the home lab side to giving people that confidence that if there was a policy change in the future at some company someone besides to change the way true nasa's produced and decided they should have a registration one the project could probably get forked and become unpopular with two the version at least you have now will not do that it won't change the version you have um that just makes me feel a lot more comfortable um i i think a lot about uh and i know this is a popular topic among if you drew a venn diagram for people that are privacy oriented and home lab builders there's a pretty good overlap of those people because we want autonomy over our data we want to decide what's done with it we want to decide how these tools should be used and you know i i believe i have the right to keep those mavica photos and not necessarily have to put them online you know unless i want to and if you think about it your decision on which nas to use is not different than an enterprise decision on which nas to use because if it's an enterprise they're going to get everybody in a boardroom or a meeting room and say okay what's important what do we want what are the features that we're looking for who are the vendors or you know that kind of thing and then you as the home lab person you may not have a whole team of people that are working with you on it it's your decision that's also one of the benefits of home lab because you don't need the ceo's um you know blessing on anything but you still have to consider like what's important to you as an individual how private is your data how bad would it be if it was gone how bad would it be if it was accidentally exposed and do you care that there's a company that's on top of that solution that some cpu cycles are sending information their way does that bother you and you as the individual will make that decision because it is your decision to make and that's why it's a debate because there's no clear one way to go there's all these different factors that you have to choose between it's just like an enterprise they do the same thing now let's get on to a couple zfs myths i i think this is um i i have a uh if you type in like zfs torture testing i have a video on my channel about this where we show how portable it is and how resilient it is um but that also includes the fact that no their ecc memory is always better don't get me wrong i'm not saying you shouldn't use ecc i'm saying you do not have to use ecc and this is what people keep trying to say is that you have to use it or you'll end up with bit rot and it's actually not true um the the level of integrity checking before a copy on write happens means that even if something in memory got flipped around an ecc is supposed to correct for that error correcting memory should correct the error and it is better that way it does do the error correction but if you don't have ecc if a checksum is reached when it tries to write something to the drive and the memory has some type of brokenness in there the likelihood that the memory would only be broken for a single bit inside of something being written to the drive that would somehow then also pass a checksum because each copy on right has a checksum to make sure and verify that every right happened um is just you go play the lottery um you're more way more likely to win the lottery than that to happen and i've covered that by shocking and pulling memory out of a running motherboard showing there is some challenges now granted if you're running a scrub um pulling the memory out can potentially cause problems not because of the ecc because you're taking a pulling memory and writing it if you did it enough and you lost enough pieces of a journal yes eventually you could break it but you it's not an absolute you can run a reliable zfs system also of note in the last 25 years i've been in tech memory errors do not account for as many problems as they used to remember it's just more reliable now so um i i yeah you know as much as if you can afford it and if the option is there to buy a higher quality ecc memory not stopping yeah there's nothing wrong with it but it's not an absolute requirement you can build a system with without ecc and it still works yeah and that's about pricing too that that's one thing i often tell people if they're buying a laptop a server or a storage unit um they might say and this is this is valid you know but but if i went with this memory this feature or whatever it's 200 more yeah it is you have to buy it right now could you save the extra 200 in a couple of months then you wait it out if you can and you can just save up some more money on top of it and trade it more as an investment i think that's valid why not because it's important so i understand yeah i have the money right now especially with the stimulus checks coming we've got to spend it all i'm not saying that's what anyone's going to do but um if it's literally down to a couple hundred dollar difference just wait a little bit longer until you save that extra money and make sure you're getting all the features that you want and you're not skimping on something that you really could take advantage of and i'll also say uh the the other myth related to memory that just comes up persistently is someone said for rule of thumb you need x amount of memory for each tv of storage that's a lot fuzzier that's not exactly true there becomes a point of diminishing return so even if you have a 500 terabytes or more server does not mean you need a roughly um 500 gigabytes of ram that's just not um what the way it exactly works once you have the operating system overhead yeah it's like if you can afford more memory don't worry yeah for sure yeah so the nice thing is if you um if you have memory once you have the operating system covered and i even have one of my systems i'm running i've done some review on is one of the trunas mini x's that is running with eight gigs of ram but 40 terabytes and yes eight not 16 gig not 24 gauge eight you heard that right and it works quite well because one i'm not sucking up memory with a bunch of jails because there's not a lot of memory to suck up for jails uh the other thing it's not doing is much cashing now the advantage you have with memory in zfs there is a certain amount of overhead it does have once you get over 16 and 24 gigs if you look and they've done a nice job and trueness of presenting this better because i've seen people in the past thinking that they had a memory leak when they look at the memory going oh my gosh it used up all the memory i had 32 gig then i bought 64 gig and it used up all that too zfs is um not wasteful of memory it actually goes well i'm gonna cache it i don't know if you'll ask for this file again but you asked for it once let's cache it and you ask for it again i'm not doing anything else in memory let's cache it you get a lot of speed advantages and actually when it comes to caching this is somewhere your money can be more well spent is in having a little extra memory in the system versus a caching drive one of the there's a lot of calculations that i won't get into is i do have a whole performance zfs that is a long talk with a lot of write-ups about how to dive into performance because yes you can also add a caching drive you can also add a log intent drive um those do help in certain ways for certain workloads and then they got better because now they have metadata drives i haven't done a video on that one yet but i believe window from level one text talks about zfs metadata drives he's got a video on that another deep dive it's like a 30 minute video because there's a lot to do um but back to just the memory topic yeah you can actually um depending on the workload and depending on what your needs are you don't necessarily need to have a ton of memory to get started you can start out with a little bit and in the case of my one at home with eight gigs of ram that runs a 20 terabyte storage for my son for all the steam games so his computer connects via iscsi to a true nas and it also uses i use it for a few other things around the house just to store some data at home backups and things like that off-site backups but uh yeah it makes for a great 20 terabyte server and it actually performs quite well with one a one gig interface and eight gigs of ram and it's a dual core system yes dual cores not quad not we're talking it's an atom processor if you type in steam freenass or steamshernasque you'll find my video on it and it actually it's kind of a proof of concept where um if you want to get into true nas it's not the barrier for entry is enterprise hardware um you can do build on some pretty basic and still get reasonable performance out of there and of course because the same truenas software that i set up on a 50 000 server for a movie studio is the same software you have access to to run on here you can still really learn the functionality you can dive into it and get good at understanding how those storage concepts work we lost you jay your sound went oh gosh my sound went you know that that that mute button uh yeah nowadays but no i mean it's a it's a great point and especially like using something like this for a steam cache i mean that's that's just pretty smart and yeah the steam cache uh well as i say cache it's actually just a steam library where everything is stored um but yeah because now he instead of putting a bigger hard drive in there he's got an nvme boot drive and then a 20 terabyte iscsi presented as a hard drive to windows so because they'll steam games yeah it may as well i mean if we have more steam games than we could logically ever finish in our lifetime but you know hey let's let's collect them all yeah well they keep getting bigger so that's a hole yeah even with the ps5 the playstation 5 i was just shocked how big the average game is i'm like oh my gosh i filled up the playstation 5's hard drive like this first or second day i had it yep um i think that's what we have for storage servers uh any final thoughts we have on sj um i do want to mention open media vault which i think is kind of like the one that didn't get any attention but the reason why is i don't know it as well as others again i do have a build video coming out and i i am going to keep an open media vault server on my network going forward because it you know i'm going to show everything i'm going to show storing things i'll show it on trueness i'll show it on the media vault because open media vault can have a lower barrier of entry than uh trueness for some people and it can get you started easier too and it's a one command install you just installed debian or you know what have you and um in my video it's going to be a raspberry pi so if you have a raspberry pi and yeah i mean you're going to have some slow downs it's not going to be the fastest thing in the world but it might be fast enough for you and maybe that's just good enough so open media vault is something i'll become more proficient with as i learn it but so far um i do like it i and i've enjoyed my time with it so that is an option that i think people might want to look into as well you know i i think uh too there's probably a separate episode we would do um because it's not like the interesting use cases for those people that are going you know i just you guys talked about a lot of speed and a lot of fancy stuff but um open media vault on raspberry pi i've seen a few different uh people doing builds on that and what's interesting to me is like you can now in general build a really low power there's a couple companies selling some kit that allows you to throw a few hard drives together with a raspberry pi and make something that is is create lower wattage and probably synology and run something like open media vault and by the way someone may ask can you run true nasa on something like raspberry pi and the answer is no not at all it's just not compiled to run on arm but open media fault does have an arm option so it is it is more of a micro home lab is is there another series called micro home labbing i don't know because if we say that too many times people say yes you need to do that too yeah you need to get on that i'm just gonna say micro home labs are a subset of this so when we do uh some home lab on raspberry pi we'll we'll add the word micro home lab in there because that's probably a good way to describe it that being said i did just buy 10 i'm not kidding 10 as in 1 0 10 raspberry pi 8 gig boards that is and that is shipping to the studio and it's going to be a 10 node raspberry pi kubernetes cluster and of course i'm going to show the power usage after i build it it's going to be in a cluster case that actually has 12 layers on it i'm going to have one as the um you know the master node and then probably a storage one as well because it's it can slot 12 raspberry pi's as ginormous i can't wait to do this video so as soon as this stuff comes in i'll have my own micro home lab i guess you could call it that you don't have to have kubernetes on each single node i mean you could have like a five node kubernetes cluster and then some other pi's that do other things and you know or not even kubernetes at all and just have individual servers on each and they have web server as one and your storage server as the other any way you'd like if there's a i really think you know raspberry pi's are going to get even bigger i think people are going to be competing at some point with what they can get away with on raspberry pi's with as little power as possible and i i think that's a great thing to try to do yeah i think we will do an episode on that just uh throwing that out for the audience because i is is like jay says one of the lowest cost ways to get in there um you can learn some of these functionality learn some of these things right on one of these raspberry pi's and it's way cheaper than about any other hardware to get started we actually i won't digress too much into that we we also talked about that in the virtualization at the end of the virtualization show we have similar discussion about building stacks of systems with raspberry pi so i think it's going to eventually just lead to an episode about them so i i could see that happening yeah so all right well um thank you guys everyone who showed up there's quite a few of you here on the live stream looks like we had uh 333 people if everyone could bash that like button if you're here if you're listening to this when we get the audio um published and we have the podcast done that'll be at um the homelab.show we the domain has been purchased if you're going there at this moment i'm saying it on march 17th there's nothing there but yes we're working we'll buy by the next episode we should have our podcast out so we're we're kind of both pulled in a lot of different directions yeah me and work it's like oh new shiny no thing yeah thing and somewhere in between me and jay both have uh jobs where we work in the tech industry so there's between the content we produce this show and then everything else it it's a lot and we gotta find time for games you know oh yeah we're sure yeah so thanks everyone for joining oh a good i 123 likes now thank you everyone so they're just now hearing it because i know there's a delay so if someone more of you can click that that'll be great uh we'll see you guys next wednesday we we had to move the time this time but we'll we'll try to put it back to the other time earlier in the day and we'll get the notice out sooner um there were a series of things booked earlier so we couldn't do it at the same time we did for the other ones but stay tuned and check out the website for more details thanks thank you and also subscribe to our channels oh that too yeah subscribe to our channels easy to find learnlinux.tv and well you're already on this channel lawrences.com

Channel: Lawrence Systems

Views: 45,083

Keywords: lawrencesystems

Id: v2vZ6lFhbY4

Length: 63min 25sec (3805 seconds)

Published: Wed Mar 17 2021