Which Linux filesystem is best in 2022?

hi i'm dj ware on this episode of the cyber gizmo i'm going to be talking about file systems once again and yeah we'll benchmark them and we're going to look at a new one as well stay tuned right after this [Music] so the last time i did this i looked at ext4 i looked at butterfest i looked at f2 fs xfs but uh that was it i'm not going to look at f2fs again i just didn't think it was all that great it was it didn't offer me anything that isn't in the other file systems now that is a file system that was made and built by samsung for handling ssds today i am going to be looking at opencfs and we'll add that into the mix and we'll talk about some of the other things that i'm going to do too so butterfs as you know is a file system that has been gaining a lot of interest in the linux community of late it started out in 2007 as a project by oracle they built the file system for linux that's what they did it for they did mark it as stable in 2013 they put it out there a bunch of people used it didn't work blew up caused all kinds of problems and data loss and they didn't retract it they just kind of left it out there and then they patched it tried again no it's still broken patched it tried again still broken it was to be honest it is kind of sad but uh it was a kind of a laughing stock for a long time in the linux community people say you run butter fs that people would laugh but it does offer some intriguing features that it offers copy on write it has pooling and snapshots checksums it has multi-device spanning although the way butter fs does multi-device spanning is the weirdest file system layout i have ever seen in my entire life and i have seen some weird ones if you want more about it anantech has a really good article on butter fs that goes through all the sort of details about the weirdness behind their multi-device spanning and their multi-device support if you're using raid 0 raid 1 or raid 10 it works great raid 5 and rage 6 native to better fs has an issue and it still has an issue as of the build for linux kernel 5.16. if you go out to their project site and i'll put a link in the description below you can see that the devs have still marked it as unstable now to me if a developer says it's unstable that generally means if you're going to try to put this in production workloads it's your fault if it blows up and you lose data because you've been warned you've been told the problem is is there is a right wormhole that occurs during a power off or a loss of power for either the device or the machine that's hosting the raid if that happens then there just happens to be a right in progress you could lose the integrity of the raid so that's not a good situation to have particularly with production workloads and large file systems if the devs tell you not to do something you probably ought to listen to them ext4 is the default journaling file system for linux distributions it was originally written to support luster i think cluster file system cluster file systems which was the company originally associated with lester i believe they're the ones that started the initial development on ext and then of course the ext has evolved from two to three to four which is where it is today and of course we all know ext4 is usually the default file system on most distributions not all but most and it also supports metadata check summing which was originally in lustre before they had metadata servers was the place they stored metadata in fact you can still use ext4 in gluster cluster doesn't have a metadata server you can use the xt4 in cluster although i think the metadata is a little bit limited there's some limitations in it because of the size in which you can store things xfs of course is next and that is a high-performance 64-bit journaling file system it was designed by sgi for their irix uh unix like operating system that powered their 3d rendering engines and also high performance computing engines it xms was designed for large files and having many people go after it so it has two very good things handles large files and it handles parallel i o very well it is also journaling and journaling unlike ext ext4 you can turn journaling off x of s you cannot but xfs you can redirect the journal to another drive if you want open zfs is a 128 bit file system and it comes with a logical file manager to allow you to do setup and expansion setting options what and what have you but uh open cfs is a it supports replication compression encryption snapshots uh and and some rudimentary i'm saying rudimentary data protection uh and some call cfs copy on right that is incorrect oracle doesn't has never referred to zfs as copy on right it's redirected on right and there's a difference in the way the two work you want to find out more about it go over to oracle and read their zfs documentation and how it works all tests that i conducted are on hardware today i'm going to be using an amd ryzen 3 700 with 32 gig of ram i'm using a samsung 970 evo that's a 500 gigabyte uh nvme i'm running on pcie 3. so i'm not running any of the newer ones um that are on pcie 4 so yeah that machine just doesn't have pcie 4. i'm running debian 11. that's just what that machine runs and i'm have no reason to change the operating system even though it may or may not have the back port patches for butter fs but you know i'm not testing raid so it's not important to me uh open cfs defaults were used so there's no compression and no encryption involved in it so butterflies same thing use the defaults for this benchmark xfs same thing ext i did use the defaults as a journaling file system and i also could fit made another pass where i turned the journaling off just to see what the differences in performance would be read as you can see zfs does quite well in this particular benchmark you'll see that there's some erratic behavior there with particularly the three and four worker loads this benchmark is an i o zone benchmark i've used it before the methodology is published in their manuals for io zone uh the work the worker load is simulating concurrent users so this would be one one user then two concurrent then three four and five as you ramp up what you're looking for is erratic behavior like what you're seeing in zfs that usually indicates some kind of configuration issue you might see it roll off that could be a memory or a buffering issue or it could be that maybe the file system is just not keeping up and your drive isn't fast enough for the amount of io that's smart that's swamping it so but in that case that doesn't indicate anything other than a perhaps a configuration issue because it's erratic and the rest of them are are down quite a bit slower than zfs is you might ask wait a minute that's faster than the drive and you'd be right it is but zfs does a lot of its work in memory so that's why it is so quick f read is a c program f read or a file read so this would be common to the unix kernel also to any of the utilities that come with your machine that are written in c unless they use something other than standard i o and that is possible so yeah it's likely that you would encounter that on c applications so zfs does not do well in that workload but xfs ext with no journaling does well and the standard ext does well in that workload as well you'll notice that butterfs does not do well in that workload p read is a buffered read zfs working good the rest of them kind of the same with the xt4 last you're going to hear that over and over again f right again same behavior zfs not doing too well although it is more of a ramp up so the issue probably is a configuration issue with the read so yeah the writes seem to work oh i mean they're scaling up good they're just not scaling out very well xfs is doing good ext no journal is doing a little bit less but butter fs does pretty well here and ext4 again dead last but you know you are this is where the journaling is coming into play um as far as p right you'll see that again here was the fs performing erratically i would consider that probably not erratic it's probably rolling off because you don't see a spike on any of the workloads that are heavier you see a gradual roll-up after after you get past three xfs is not as performant as the xt no journaling is in this particular workload but ext is gradually rolling off whereas xfs is gradually rolling up yeah so it's gradually scaling up but refess does quite well in this workload i wouldn't say that is poor it's doing as well as xfs seems to be doing uh it's close anyway ext4 that last re-read and this is a as re-reading the same area of the disk or the file and zfs does well in those workloads uh ext no journal does as well although you'll notice there's a hitch in the worker too by rfs it it ran it scales up but not to the extent xfs and ext4r and ext4 with journaling dead last this is the rewrite so this is rewriting an area of the file over and over again now the erratic behavior is back but to a lesser degree on the zfs but overall the performance is good again that's probably pressure from memory ext4 no journaling it's pretty stable above three users two users yeah it's pretty stable but rfs is doing pretty well here is it's comparable to xfs and ext4 did last random reads this is randomly reading a file i guess the only thing else to say here is that i think ext4 node journaling is performing slightly better than exit fast by rfs has some issues ramping up and notice on the smaller workloads like a single user workstation it does not perform it very well it isn't as even as performant as the xt4 is xfs on stride read now that's that's reading an area of of the file jumping ahead reading another area jumping ahead now that could be random it doesn't have to be the same blocks it could be random that would be indicative of a database doing a sql search we're kind of seeing the same behavior in xfs and ext they look close to me they're i mean there's a slight difference in the scale up ext with four with no journaling seems to scale up faster but we still have that hitch that workload too butterfest is scaling up but again doesn't perform as well on the slow on the smaller workloads random writes uh zfs is doing well here and but it's rolling off pretty quickly x and uh xfs it's a little bit slower than ext4 and i think ext4 is actually ramping down but it's ramping down to the same point where butter fs is ramping up and then ext4 dead last reverse read this typically you'll find this in compilers particularly multi-pass compilers where they'll do this and that's reading the file backwards you'll find in this case the fs is doing quite well ext is doing great for this workload although it again it is it's reached its peak at around three users it don't seem to gain anything beyond that so that could be it's just that's as fast as it can go xfs it's climbing up a little bit and then it stops butter fs starts out slow climbs up but not as well as xfs or ext and of course ext journaling again dead lab initial right this is creating a file on disk for the first time zfs again is doing quite well here it rolls off very quickly but you'll notice that zfs does well even on single user workloads ext4 does quite well butter fs does quite well here xfs not as well and ext for dead last so i wanted a mixed workload here um is also part of the there's 13 different simulations that i ozone can do all 13 of them i would have to say xfs and butterfs are running pretty close together uh ext4 no journaling definitely has an advantage on the lower workloads on the smaller workloads ext4 again but dead last as far as the geometric mean is concerned this is kind of bringing all the results together and then deciding you know who wins who had the best overall performance and zfs no question xfs ext4 i would say they're close and maybe not close enough for you but they're close to me ext4 dead last no question about that and butter fs would be second from the bottom and that's the way it runs out so cfs what do we conclude does is the fastest in every every test except for f read and f right that should be f right ext4 slowest in every file system test except for f read and that should be f read there as well ext no journal and xfs they're close there's some differences in in the workloads but up in the higher workloads they're pretty close and then butterfest i would say in some cases it's good some cases it's not so good uh it does not seem to handle smaller workloads so if you're putting this on your workstation you're probably on having you're probably going to see poorer performance than you would using one of the standard file systems it's just not as fast as dfs it's not as fast as ext4 with no journaling and it's not as fast as exit pass what's the file system is best i would say i mean just just my opinion uh cfs would be a good choice for non-root file systems i don't i wouldn't recommend it for your root file system however because of the you're going to encounter f reads and f rights and then that's not where it excels there are better choices for that for nvme to prevent wear leveling i i don't know if i can really say this is a recommendation but it's a choice right you could turn journaling off and that would save your ssds some right some lifetime so as far as xfs it's a good all-around file system as far as trim support i mean ext has had trim support for some time xfs has it as well but you might want to check and see if your device supports fs trim to make sure that that it will that it will preserve otherwise if you can't find that you can run trim manually there are utilities in xfs that allow you to do that butter fs is kind of a buzzword we've said that before it's kind of the shiny object that everybody's running around thinking it's the greatest thing since sliced bread however in the tests i would say it does not have the performance level that the older file systems have in linux workloads and workloads that you would encounter in small business medium business and even enterprise workloads raid five and six there is a code there is an issue that is being addressed root file systems what do i use i use uh ext4 on x86 platforms i just do it for convenience i know that it's not the most performant but it works well enough for me in most cases now i do leave journaling turned on even though i know probably shouldn't if for my workstation here for this workstation behind me that is xfs that's running xfas on that one root file systems on arm i use xfs because they're limited memory gluster wants to use xfs for its default file system i use zfs for everything else i use the fs for my raid i use it for my nas i use it uh to store videos i use it to play back videos and edit them i use it for everything and i do run zfs rays as for butter fs yeah i've tried it recently yeah so don't accuse me of not having used it since 2013. i have i've tried it uh it just doesn't have any place in my workflow because of the performance the boss that i work for it was it was old school old school engineer i mean he uh he had worked on systems that um back in the 50s 60s and on up and uh he if you came to him with a new idea he would listen to you and he would entertain it but he would never implement it until you had proof of that your idea was faster so if you wanted him to say yeah okay you've got you got some budget to go and do that that's that's all well and good but if you went to him with an idea and you had no data he would you say well that's nice come back when you have an answer to my question does this improvement performance because if you didn't have proof he wouldn't accept it he would always say that opinions are worth nothing in this business if you can't prove that your idea works don't bother me with it and that's the way it was and that's the way i'm leaving this today if you if you hear people say that butterfs is the fastest file system um you might want to check that benchmark and see how they're measuring it hope to see y'all again very soon and bye for now [Music] you

Channel: DJ Ware

Views: 50,113

Keywords: DJ Ware, CyberGizmo, Linux, Filesystems 2022, btrfs, ext4, xfs, openzfs, zfs, iozone, benchmarks, performance

Id: G785-kxFH_M

Length: 19min 32sec (1172 seconds)

Published: Wed Feb 16 2022

Reddit Comments

[u/AEOlovesFascists]

This seems purely from the perspective of workstations or servers. Not sure how much value that has for this sub and its regular desktop consumers.

[u/HardwareRaidIsDead]

I started using xfs in my VM's and it has been lovely so far the performance was great.

[u/binary_agenda]

I've looked at a few of these file system reviews and they all claim a different file system is the best file system. This is the first one I've seen include ZFS. Most people talking ZFS are about enterprise level stuff for your home servers. Are there real advantages to using ZFS on a single drive for the average gamer? ZFS for /home?

[u/GeneralTorpedo]

10 months old video

Brotherman...

[u/Due-Word-7241]

he compared RAM and disk, no real benchmark filesystems to the disk. zfs uses cache on RAM, that is why it is fast.