How to install and setup MergerFS

hello and welcome to another video in this video we're going to talk about merger fs and snap rate merger face is a union file system that lets you pull multiple drives into a single Mount Point compared to something like raid merger FS doesn't calculate any type of parity or any backup but it has a couple of advantages but the advantages of using something like merger FS versus raid is that it's very easy to use it allows you to use multiple size drives so you can have a two terabyte a four terabyte and an eight terabyte and pull them together so this is very handy and it also has the the the advantage that if you lose one drive and you can't rebuild your array you only lose the data of OneDrive you don't lose the data of your whole array because in raid like let's say raid 5 if rate 5 protects you from one drive failure if while rebuilding the drive your another Drive fails then you can't rebuild your array and then you lose all your data merger FS doesn't calculate any party or any type of backup but to counter that we're going to use another software called Snap rate now snap raid it's software that only does priority calculation but it does it on a file system level it doesn't do it at a block system level like like rate now another cool thing about merger effects is that is that you can remove or put new drives into the array without having to rebuild the whole array so if you one day you want to expand your storage you can just buy a new Drive put it into your computer add it to the merger FS pool added to your snap rate and then that's it so here I have prepared a demo and I'm going to show you how to set everything up the demo that I have set up for you is going to assume that you have at least three drives so that we can have two drives for data one drive for parity we're going to use the same VM as before and we're going to log in with SSH now I'm going to type in the IP address of the machine which I believe is zero to four seven perfect okay so the first thing that we need to do is we need to create partitions for our drives so we're going to use lsblk to list our block devices on our Linux VM and as we can see here we have our boot drive with all our partitions and then we have our three block devices or hard drives that are 20 gigs each we're going to use the utility called fdisk let's start with the first drive which is sdb here so we go f disk forward slash there forward slash sdb so this is an interactive utility this utility is non-destructive meaning that all the changes that you do here is not going to affect the drive until you decide to write your changes to the drive so let's start with creating a single partition so if we Type M for the help file we're going to choose G to create a new GPT partition table so we go g and enter so we have a fresh partition label now we're going to press n to create a new Partition hit enter partition number one it's fine first sector default last sector it's going to use all the drives that's what we want if you want to split your drive into multiple partitions you can also do that but for this one where I am assuming that you want to use your entire drive this could be a two terabyte a four terabyte drive in this case it's just 20 gig Virtual Drive so we continue okay so now we have a new Partition of the type Linux file system and up until now not we haven't done anything to the drive but now we want to write our changes to the drive so we're going to press W and then enter okay so if we list our block devices again LS be okay we can see that now we have a partition here sdb1 with a size of 20 gigs we're gonna go ahead and do the same thing for the other drives so that we can have three three partitions on our three drives okay so we're gonna do f disk now it's going to be SDC same thing we press G for new partition table we press n for new Partition leave all default and then press W to write your changes and now we're going to do the third Drive which is ssdd same thing G new partition table and for new Partition leave all defaults and write your changes to the to the disk now if we list again so now we have three partitions sdb1 SDC one and sdd1 so now what we're going to do is we're going to put a file system on them I chose xfs it's my preference I like it it's the default for Rocky and red hat and there's nothing wrong with it you can use ext4 ptrfs xfs anything anything you want so let's go ahead and do xfs to do xfs file system on a partition we're going to use the utility called mkfs we're going to do Dot xfs and then we're going to give it the path to our partition so in this case it's slash Dev slash sdb1 and then just hit enter okay so our file system is created we're going to do the same thing for the other one SDC one and s d d one if we list our block devices again there's nothing that changes here but we now have a file system on this partitions so now what we need to do is Mount this partitions this file systems in our machine so that we can access them and we can write and read files from them I'm going to choose the directory slash MNT and we're going to create some folders there so let's navigate to slash MNT so as we can see there's no folders here so we're going we're going to create a couple we're going to use the utility called MK there and it's going to be just give it a name to our folder disk one we create another one disk two and we're going to create one that is going to be called parity one like a traditional raid system you have to sacrifice one or two drives depending on how how much protection you want just to store the priority data in this case I'm going to use one drive for parity so this will be the equivalent of having a traditional raid 5 on your system so if we list our folders now you can see we have this one disk 2 and parity so now we're going to mount our partitions into these folders for that we're going to use the utility mount we're going to select the partition that we want to mount so in this case it's going to be slash Dev s d c s d B1 and we're going to mount it in disk we're going to mount it in parity one so our first drive sdb1 is going to be your priority Drive now we're going to mount our second Drive which is slash Dev sdc1 this is going to be our first data disk so we're going to mount it into disk one and our last disc sdd our last partition is going to be mounted into disk two okay so if we list our block devices you see nothing has changed but now if we list our Mount points with DF h you can see that we have three Mount points for our three drives okay so now that we have our Mount points and our drives mounted there so that we can access them and like write files to them and all that good stuff so now it's time to set up merger FS the first thing that we need to do is we need to go to the merger Fest website or more like their GitHub and download the RPM installation package so that we can install merger FS so we're gonna go to our browser and we're going to look for merger FS so you can go to the GitHub page in this case I am running rocket Linux 9 so I am looking for the RPM package if you're running something else like Ubuntu or another distribution of Linux then you could also follow the steps here to compile it and install it but luckily for RPM there's already a package so I'm going to go to tags and then release this and as you can see here we got a nice RPM packages in this case there's not a specific package that was built for Enterprise Linux 9 or rocky Linux 9 but I used Fedora core 31 and it's working fine so I'm just going to right click on here and copy link address I'm going to go to my VM let's go to slash opt or anywhere you want and we have wget installed no so we need to install that with dnf install so what you get so that we can download web packages okay yes except okay now we can do wget we paste the link and press enter going to go and download that package and if we make a list here we can see that we have the merger FS RPM package here so I'm going to use dnf just to install it like any other normal RPM package so dnf install merger FS it's going to look for the dependencies this fuse type of file system so it's going to install fuse and fuse common that is perfectly fine click on yes to install and that's it now we have merger Fest installed on our Linux VM now we're going to set up merger FS the prerequisites for setting up merger FS is you need to have your drives partitioned and mounted we already did that so we're good to go so there's two things that we need to set up to make merger FS work we need common Mount point or where we are going to mount all our Drives together we also need to configure our FS tab so we're going to navigate back to our Mount folder or MNT folder so let's go there yes we have our drives here and we're going to create a new folder here this folder I'm just going to call it storage but you can call it anywhere anything you want this is just the place where all your drives are going to be pulled together okay so now we have our storage folder that's perfect okay so now that we have our storage folder created now we need to gather information about our drives and our Mount points so that we can set up our FS tab file so what is the fs tab file basically this file is very handy because this is the the configuration file that the Linux kernel looks at when it's booting up to bound all your drives so this includes your boot drive your EFI drive and your system drive we want to put in this file all our drives and our merger Fest pooling Mount point so to gather the information of our drives we're going to just use the the ls command and we're going to list our Drives By ID so what is the ID of a drive the ID is basically the hardware ID that identifies your drive and the reason why I like to use this instead of the traditional label that Linux gives to your drive like SDA sdb or SDC it's because if you remove drives or add drives to your system the Linux how the Linux kernel labels your drives is basically how in the order that they're being detected when the system is booting up meaning that these labels could change it happens very rarely but it has happened before if the label changes just on your drive then now your SDA Drive is the eight terabyte drive instead of the four terabyte and now you're mounting this drive into the parity Mount pawn and it's just very messy what I like to do is I like to use the disk Hardware ID instead so to find out your Hardware ID we're going to use LS and we're going to go into slash Dev slash disk by ID and as you can see there's a lot of stuff here but we're going to make our life easier by using command called grab is basically just a filtering and searching utility that is very handy to filter out information that we don't need we're going to use graph so you just need to type the pipe command here and then grip and we're going to look for a first drive our first drive is sdb1 well in this case we're looking for the partition not the drive so sdb1 as you can see here is the sdb1 partition so we're gonna grab the first one here ATA virtual box hard disk drive yours could say something like Seagate or like Western Digital or the brand of your drive and followed by a couple of serial number or something like that and part one so it's important to to use the part one so that we use the correct partition so we're gonna copy this one okay so this is going to be our parity one perfect now we're going to look for our second Drive which is SDC one perfect we grab our ID here this is going to be disk one and we're going to go and check for sdd one same thing we grab our ID and this is going to be disk 2. with this information now we can configure fstab configuration file we're going to use Vim slash Etsy slash fsw okay so this is our configuration file now here is where we're going to determine our amount points and how we want to mount our drives let's start by just configuring the mount points of our current drives like I said before we're going to use the ID so we need to give the full path to our drive so we're going to grab slash Dev disk by ID we press I to insert a new line and then we paste so now we're going to grab the ID of our first drive the parity one now we do a space and then now we need to tell it where we want to mount this partition like we said our first drive is going to be our Priority One so we're going to go and put it on MNT slash parity one now we need to give the the settings and the options that we want for this Mount points so it's very easy we're just going to copy what we have up here for our root file system our file system that we created previously is xfs if you chose something else like ext4 or btrfs or something like that here's where you would have to specify what type of file system this is so in this case my file system is xfs so I'm going to put xfs now we're going to leave the default settings here and just copy the same thing 0 and 0. okay now we're going to do the same thing with our other two drives Dev this by ID we're going to grab our second Drive I'm going to paste it in there and then this is going to be slash MNT slash disk one xfs defaults oh sorry here I missed it defaults zero space zero now we're gonna go and do our third Drive they have this by ID we grab our ID here slash MNT slash disk two again this is xfs default zero zero okay for the moment we can close our FS tab file so to do that we just do Escape column right quit so WQ enter and then if we open our drive again you can see that our changes were saved now it's time to set up our merger Fest bound points so what we're going to paste here is going to be the following these are the options for merger FS we're going to do the same thing we're going to give it our source so basically this is the the drives but in this case is the Mount points that we want to mount into the pool of merger FS so for this we're going to choose in this case slash MNT so slash MNT and here we're going to only use our disk drives we're not going to use our parity drive because we don't want to store regular files on our parity drive we only want to store parity information because we named all our disks uh the same way like disk one disk 2 Etc here we can use a little wild card to to tell merger Fest to use all the all the mount points that start with disk the disk Name by putting a star here we're basically telling the system to use all the mount points that start with disk and whatever is uh whatever follows after now we need to choose a destination where we want to Mount all this files so in this case it's going to be our previously created slash MNT storage folder so slash MNT slash storage and now we need to tell it what type of file system we want to use to mount this so in this case instead of choosing xfs like we did previously we're going to choose merger FS so in this case it's going to be fuse dot merger FS merger ffs so this is just the the way it's uh the file system merger face is named because merger FS is a fuse based file system so this is how you need to configure it so now in this part of the configuration file we're going to give the merger Fest options usually just use the defaults and the defaults work really good for me and I use one of the merger FS policies that basically explains whenever you are going to write some information into this Mount point so it's going to choose the drive that has the most amount of free space and it's going to write the data to that drive and then if another drive now has more free space then it's going to use another drive basically making it so that your drives get full evenly and that you're not like filling up one drive and then filling up the next drive and filling up the next drive we don't want that well at least I prefer that so that all my drives are getting filled up at the same time evenly I'm going to paste the configuration file here you can copy it exactly as it is or so if you want to learn more about the options and the policies that you can use with merger FS can go back to the GitHub page and go to the readme so this is the link and if you scroll down here you can see that there's a description for every single option that we're setting up here there's also the policies so if we scroll all the way down here we're going to see that there's a policy section policy description basically this is going to explain all the options and the policies that you want to use so the one that I'm going to be using is MFS if we look for MFS here here we go it's called most free space so it's the description of this is basically just pick the branch with the most available free space like I explained before I want the drives to get full evenly so now let's set up merger FS going to copy the configuration now I'm going to leave this configuration on the description below so you can just copy paste it and so I'm going to paste it now there's a couple of options here that you can see like cache files partial drop cash on the close all of these options are described here on the on the GitHub page so I will encourage you to go and read a little bit about this it's really interesting like I said before my current policy is MFS let me just make this a little bit bigger so you can see okay so category create MFS most free space then I have another setting here called mean free space 10 gigs and basically what I want is at any given moment in any Drive I want to leave at least 10 gigabytes of free space at the end of the drive so if one of the drives is getting full it's just going to stop writing data and leave 10 gigs of free space this can be changed to maybe like two or five but this is just a precaution just to make sure that I don't feel the drive 100 and then the other option that I have here is FS name merger FS this is just a label so that I can find it very easily when I do the DF DH F command after that it's just the defaults like the other drive and then we're just gonna put zero and zero that's all the configuration that you need for a merger FS so we're going to write and equip let's clear our terminal so now if we do df.h R merger face mount point is not there yet so to mount that we can use the command Mount again and we're just going to select the mount storage location so Mount storage I'm gonna press enter so it gives us it just gives us an error here that says that we need to reload our demon we just need to reload the demon so that it can acknowledge our new changes but we can do it now or we can just reboot the machine and it's going to be fine this is because the Linux kernel is detecting that we changed a system configuration file but it's perfectly fine I just did a df.h command again to see if our Mount point is up and running as you can see here we have our Mount storage Mount point and it says that it's merger FS and it's 40 gigs in size meaning that it's our two 20 gig drives merged together into one single Mount point of 40 gigs whenever you're going to write data to this Mount point is going to choose the drive that has the the most free space and it's just going to continue to fill it like that so if you write data and it chooses the disk one then it's going to write to the Discord this one and then the next time you write another file it's going to choose disk 2 because now disk 2 is the one with the most amount of free space okay so this is how you set up merger Fest it's super easy to use like you see now we just need to create partitions create file systems configure FS install the RPM package configure our FS tab then you have a pooling system up and running on the next video I'm going to show you how to set up snap raid and snap rate is going to be your parity and backup solution in case you lose a drive or in case you lose two drives so whatever level of protection you want to go with I'm going to Showcase two examples of that I'm also going to give you a script that you can use to run it every day or every week so that you can automate the process of your backup and your priority calculation so I hope you enjoyed this video leave a like if you like it remember to subscribe if you want to see more videos like this and I'll see you on the next one

Channel: Distro Domain

Views: 4,617

Keywords: how to install mergerfs, mergerfs 2023, how to setup mergerfs, home media server drives, how raid works, what is raid, home server backup, rocky linux mergerfs, how to partition drive in linux, linux drive partitioning, how to configure fstab, what is fstab, how to automatically mount drives in linux, how to pool together drives in linux

Id: n7piuhTXeG4

Length: 24min 45sec (1485 seconds)

Published: Mon Jun 26 2023