FreeNAS ZFS VDEV Pool Design Explained: RAIDZ RAIDZ2 RAIDZ3 Capacity, Integrity, and Performance.

I'm here from Warren Systems we're gonna talk about FreeNAS ZFS pools raid-z z2 z3 capacity integrity performance and of course the subject of much debate and many sleepless nights for people designing storage servers if you wanna learn more about me or my company head where to lawrence systems comm if you want a hires for project there's a hires button on the top there if you would like to help this channel out in other ways or our affiliate links below that to help us out and get you offers and deals on products and services as we talked about on this channel and of course there's the forums where i'm reading this from and that's a great way to participate and communicate with me for video suggestions or just further discussion on a topic most all my videos are cross posted within these forums and this particular post is something i've been putting together not because i have the answer for you and i want to talk about that there is not a one-size-fits-all when it comes to laying these out a lot of people start with this simple question and when you first get into storage like oh how should i lay out these eight disc ten disc four discs whatever that number is when you're building out your storage pool and this is where all the balance has to be met unless your budgets unlimited you may not get what you want unlimited budget will get you quite a bit in there still considerations even when it's unlimited budget because of the loss of physics and the advancements of technology anyways so when you're setting up your ZFS pool it's basically pick two but you can't pick three you're gonna pick capacity performance or integrity and where you want to land is going to vary with that setup in that design of these ZFS pools I'm going to talk about among free now specifically but this applies to ZFS pools in general ZFS I'm a huge fan of it is an incredible system and it is highly highly fault tolerant provided it's set up properly it has performance and fault tolerance in its design and it's used at scale very much commercially because a few people have asked is this even something that's used in the enterprise absolutely enterprise companies don't always reveal what they're using but I will at least preface this with yes we have even consulted with very large companies that do not want us to drop their names although be kind of cool to say if I could um yes the is used in the very commercial markets all right now what you have to think about here and we'll do a little bit of background so how is a ZFS pool formed in in the basic sense if you're setting up a simple system yeah you're probably going to have one V dev but once you get into larger more enterprise systems you're gonna have multiple V deaths and that's going to affect your performance so once you're creating these the basics of the performance in ZFS drives our logically group together into one or more V DEP each feed EV can combine physical drives across the number of raid-z configurations if you have multiple V devs the pool is then striped across the V deps so you want to make sure there's fault tolerance in your vidos because if we lose this feed up here and we didn't have proper fault tolerance in it the pool falls apart now the V devs are the storage parts of a ZFS pool they are separate again and I'll leave a link to this so you can read about this and we're not going to dive into a bit more reading of course and that's what this whole point is to give you all the tools and links to dive into this and learn more on this topic but the CFL's ill and slog demystified is a great article over here Dyke systems and a few of these are attack systems they spend a lot of time developing the ZFS and working with it to not just develop FreeNAS but they do contribute to the entirety of the open source project that is ZFS and of course because they are doing all the support for it they know a lot about it because they do that enterprise level support i'm obsession when you're designing these storage servers so get back on topic there are other pieces that you attached a pool such as the cache drives and the zero intent drive so you can also enhance and boost performance but the storage the data lives within the V devs the cache and zil are something separate and there are ways to buffer the commits or cash out frequently use files and then you know quickly get them to you and ZFS goes a step further and also will use the memory to create large amounts of cash and once you show that real quick here mike is not doing much so we actually have the ZFS caches 28 gigs of caching going on right now that will help serve up files faster in this particular machine whirring talk about is my video server so we'll get to that in a second six metrics for measuring ZF is performances on first articles I linked in there and it's really a balance like it shows in a very beginning read I ops per second right I ops streaming read streaming right speed now I ops is tricky so which when you want a lot of I ops and let's say a database application would be an example and we'll go more real-world let's talk about a WordPress website let's say we need a wordpress website that has a lot of reads because there's people pulling all these you know articles across there so we want those articles to be served up quickly it's a lot of database read we need as high read I ops because all these little files have to be served graphics and everything else and we need them to get there fast we need these queries against this database to be fast because WordPress back-end is all database driven so maybe someone's querying and searching through old archived articles right i opps well do you have articles being posted to it very often no okay so maybe you sacrifice a little bit the right I ops because the integrity of the site means more to you the uptime and not losing drives that serve this site are real important so you sacrifice some of the right apps because you added some integrity and more capacity to the system in terms of how many drives you can lose so there's you know a factor in there what about streaming read well same thing we want to be able to stream really fast because maybe we've embedded some videos in it website so now we have like a clearer well maybe somewhat concise use case on there what if you swap that over and say what about a forum like the discourse forums I run well now we need read I ops and write apps because we don't want it to pause want people to put articles on there and I have to wait for the article to get published or timeout while trying to publish it because there's so many read I ops and then streaming speed well maybe that's less important you know because we wanted to stream fast enough on there but you know they're mostly just reading static content regenerated without videos embed it's just whatever a screenshots someone like Tom posted in his forums so these are things that you have to consider and of course it goes further out if you start talking about running virtual machines on here does that virtual machine have to read a lot write a lot what should the cache be on there and maybe the cache drives would help maybe the cache tries won't help because well especially with a VM you're not serving up the same files time because it's very dynamic because of the way it saves it in the I scuzzy format and now you can see why I'm going to stop talking and leave you links to lots of these articles because these are the decisions are why I can't just answer the question how do I set up my drives really you have to think about what you're gonna use and their breakdown a lot of great details with all the graphics how this goes across one way to a mere six by two way mirror et cetera et cetera then they have a part to dive in a little bit more performance and some of the topics on this and then we have this site here which leaves some excellent examples now you can extrapolate it out because they started with onedrive give you the performance of onedrive this is obvious a little bit older of a drive then they break it down to this four terabyte drive I know four terabytes getting to be small here in January 2020 you're probably building this with bigger drives or maybe you have a bunch of four terabyte tries but you can see the performance differences by just changing things around from a stripe to a mirror to breaking them down into 2x straight Mayor's breaking these sound into different B dubs and seeing the performance differences you get and of course the capacity sacrifice so this is a real way to visualize that so as you gain fault tolerance you lose capacity but you do get certain read performance benefits but then your rewrite performance goes down to here and this is the write performance on a raid z3 versus the read z2 of the same drive so anything you did was change the Z type and you can see the differences that created in read and write performance and of course capacity loss there's a lot to think about in there and of course this white paper layout rate from Ike system is a whole white paper on storage and servers driven by open sourced by Ike systems back to the whole pieces and all those examples of real-world workloads another great article and all this is linked in my forums so you guys can dive deep into it and of course this is a kind of a nursing article written by Matthew the populated open Z has spawned a great community of user system and architect developers computing wealth of advice tips and tricks and rules of thumb of how to configure ZFS and talking about setting up like wide ZFS and there's some you know simplicity you can have to having a wide ZFS pool but of course that stresses the pool differently depending on how many wide it is and a lot of this the reason for me posting this video one I get a lot of questions that people asking about it to get her work on a review of true Nass so when you look at my system here and we look at how my discs are laid out this is the server that mostly stores my videos and is backed up completely into another system so I just built a really simple raid-z pool with these drives right here so I took this raid z2 offers enough fault tolerance for me and my videos because I know it gets backed up every night so you know in theory if I lost a drive I could resilvered and okay while that resilvered process does tasks the system raisi to gives me enough fault tolerance that i could possibly survive another drive being lost while i'm doing it and hopefully i wouldn't lose a whole day of videos that i stuck on here so I chose rate Z to this layout works really well the other things stored on here a handful of VMs from my lab server once again they're not anything that's using I apps if you watch it in my lab videos I'm doing a bunch of FreeNAS talks I'm doing a bunch of talks on PF sense and those type of videos are restored here and of course those VMs are stored here those VMs that are stored on here once I boot them up for the demo with a couple Linux servers that I have on here they're not doing anything that requires a ton of rewrite intensity so read Z 2 with nothing else added to the pool works perfectly fine when you talk about enterprise usage you talk about you know real intense systems that are going to be pushed to the limit we talked about true Nass being awesome commercial option that's why I'm doing a video on it but how would you lay out a large true dance server and I've already done the basics going over the hardware well this is a series of raids II - each one of these so there's five drives in each one of these raids II - and like we said the very beginning for each V dev that makes the pool it stripes the data across so we get this amazing write performance on here and the fault tolerance is Z - limited to each grouping of 5 drives then outside of there this is that real world example here's the log the cache drives and a spare ready to be thrown in the mix if any one of these fales so under this circumstance this is more of a higher performance layout where we can have the fault tolerance raise e to we group it down to these drives so any one of these vetoes if they lose a drive we can in rebuild that particular drive and then we have the zero intention log here and then the cache cache we're serving up the repetitive files and the zou log because you may want synchronous writes per you know really good integrity when you're setting up with like NFS and I scuzzy for serving up VMs and I'll dive into the demo of this server soon but I wanted to bring this up because the question comes up so often about it and it's why I can't say where you see the first comment of it depends when someone says in a forum post you know how should I lay out my drives well it depends and this is all the things that depends on so these posts these I've read through all of these maybe even more than once and sometimes again when I'm thinking about it when I'm laying something out because I want to be right when I do it but there are great reads they're a great way to get a better grasp and more in depth on this particular topic and hopefully the under gets you a better understanding of why people start with it it depends and why you can't just lay them out because it kind of depends on your use case for it so I'll leave a link to this forum post which goes to the video which the video links at a forum post and that cycle continues and of course all these links are right here in the forum post so you can really dive deep into this feel free to contribute to this forum post if you want if you have other sites that maybe I missed that you think give a great explanation of this or that you just found really really helpful on there or if you have questions in discussion you want to keep this topic going please this is a great place to post it and there are different methodologies to doing it I'm not saying that my layout is exactly right this is something I think's important for technicians to think about including myself I'm always open to other ideas like I said I went really simple with this one this one worried a little more what seems like more complicated but can provide better performance I'll be doing that when I dive in a video but like I said this is still a great discussion about how all this needs to be done and there's a lot that goes into it that's why I wanted to make clear thanks and thank you for making it to the end of the video if you liked this video please give it a thumbs up if you like to see more content from the channel hit the subscribe button and hit the bell icon if you like YouTube to notify you when new videos come out if you'd like to hire us head over to lawrence systems comm fill out our contact page and let us know what we can help you with in what projects you like us to work together on if you want to carry on the discussion how to wrote of forums that lawrence systems comm where we can carry on the discussion about this video other videos or other tech topics and general even suggestions for new videos they're accepted right there on our forums which are free also if you'd like to help the channel on other ways head over to our affiliate page we have a lot of great tech offers for you and once again thanks for watching and see you next time

Channel: Lawrence Systems

Views: 67,025

Keywords: lawrencesystems, freenas, zfs pools explained, network attached storage, raid z, zfs performance, network attached storage setup, network attached storage explained, network attached storage tutorial, network attached storage 2020

Id: -AnkHc7N0zM

Length: 13min 56sec (836 seconds)

Published: Sat Feb 01 2020