How Much Memory Does ZFS Need and Does It Have To Be ECC?

foreign [Music] systems and how much memory do you need for ZFS and wait I know it must be you need for each terabyte of storage you need at least so many gigabytes of memory there's some formula right it's a little fuzzier than that and that answer has been at the top of Google search results for I think too long because that's not the most accurate way to do this what we're going to cover in this video is one that you don't need that much memory for ZFS but if you do use lots of memory you're going to get better performance out of ZFS I've got a couple videos where I dive into the Y which is of course caching the more memory you throw in there the better the arc cash will be therefore the better performance that video is linked down below where I demystify hopefully the ways that the cache system works kind of a side note is what about ECC isn't it a requirement for ZFS isn't my data at risk and I'm putting all of this data you know in in the hands of one bit flip and destroying all of it that's not true either I've got a video talking about ZFS being a cow copying right file system and don't worry the data Integrity is fine even if you're not using ECC is it better to use ECC that's a general yes because from a Computing standpoint having error correcting memory is a good thing but it's not a required thing in order to do this so I wanted to clear those up right away now let's dive into and we're going to talk specifically about sure Nas but this applies to GF ZFS in general when you're setting this up for how much memory you need for your ZFS system okay we're going to start at the low end this is my low end Intel atom system this is what I call my purple Nas because I had a handful of Western Digital purple drives that I put in here four of them to be specific let's actually look at the storage real quick here and we'll see that I'm only using four terabytes out of 22 terabytes that are still free so just under 30 terabytes of storage on this this just keeps backup copies of things so performance is not an issue they're backups in my videos so I'm not worried about how long it'll take to get them back but if my main Nas fails hey I got a second copy but I restarted it just a little while ago because I want to just clear it from copies I've done when backing up and putting files on it to show you that with eight gigs of memory it's still not even using I have 5.5 gigs free now if I start copying files back and forth it will use some memory and if we go and look at the services I'm using in it to get the data back and forth such as rsync I have an rsync job that talks to it I've got SMB if I need to copy some files to it SSH which is part of facilitating ZFS send so my main video editing sureness copies all the data back over to this one on a very regular basis because hey why not it doesn't take much it's only doing differential copies so this job actually runs frequently but it just doesn't take a lot of performance and as you note I have quite a bit of terabytes of storage that don't match up with the amount of memory and the system is completely functional and works well other than it being slow but it's slow because that Intel atom more than anything else combined with some older spinning rust drives now let's talk about my video editing system when you get scale versus core there's some differences specifically because scales built on Linux by default it does not use all the memory it will only use about half of it so we do have 64 gigs of ram in my video editing system because I could probably even use more all the videos and actually this is why it's under load my videos has a record them right directly to this system they're getting cached this makes it really fast when I got to pull those videos into the editing software so that comes right out of cash and because I've modified the parameter I'll leave some notes in description below to tell it to use 50 gigs of cash it's using 50 gigs of ZFS cache there's always files moving back and forth in my video editing this gets me much better performance now even though this is SSD volumes they won't quite Peak out that 10 gig Nick but they'll come way closer to picking up the 10 gig Nick when you're pulling things right out of memory so this is where having more memory does help quite a bit now let's move over to one of our production core systems at my office and this system has a 120 eight gigs of memory it's pretty much all used all the time for ZFS caching there's some Services going on because we run several S3 targets for different backups that land here and this system benefits from having a lot of memory it would benefit more with more memory well we still got 5.2 gigs for you so maybe not that much more but this is using all the cash for serving up the virtualization targets that talk to it or some of the backup and restores when I do backup restores they pull from this when I have my virtual machines running NFS this is the storage Target for them and because NFS and VMS are going to be a lot of frequent rights or frequent reads more specifically for database stuff that's getting pulled it just gets all in the cache and serves this up very fast now this does have 27 spinning rust drives in it but the cash will supplement and really make those frequently called files which is a typical use case serve up extremely fast combined with a fast processor to get the data back and forth there and this system actually performs and has the IOP performance similar to an SSD array it's not going to be a one-to-one comparison it's going to vary by workload but this is an example of where having a lot of memory helps a lot when we have clients that have large virtualization systems are using TransCore as a large Target for those we load up as much memory as reasonable to their budget budget's usually a constraint at some point for how much you buy but loading a system up with 512 for a true net system is not uncommon because it will give you really good read performance for all those frequently accessed files or in the case of like a system running 80 or 100 virtual machines across several different hypervisors and pulling the data off here or in the case of video editing we have a lot of clients in that space that are more than just a single editor like myself if you have a team of editors pulling files pulling them out of memory is always going to be faster than pulling them off the drives as I said in being a video I use true and answer as an example but it does apply to really any system using ZFS you don't have to have a lot of memory but the memory will greatly help the performance of whatever system is running ZFS so it's allowed to use more memory means better caching that caching video I dive deep into the details of that ZFS is really smart and intelligent it'll take all the memory and all those files back and forth and commit them there so any service requesting them can get at them faster but if it doesn't have them you're just going to pull off the drives and work at whatever rate that operates at depending on your system configuration let me know what your thoughts are in the comments down below or to engage me further on this head or my forum for a more in-depth discussion thanks [Music]

Channel: Lawrence Systems

Views: 45,482

Keywords: LawrenceSystems, zfs memory, zfs memory cache, zfs memory limit, zfs memory tuning, zfs memory leak, zfs without ecc memory, zfs pool, zfs file system, bit flip, what is ecc memory, ecc memory

Id: xp6g-8VS06M

Length: 6min 58sec (418 seconds)

Published: Fri Apr 07 2023