ZFS Snapshots Explained: How To Protect Your Data From Mistakes, Malware, & Ransomware

one of the superpowers of ZFS is snapshots it's one of the reasons that a lot of people dive into ZFS and the absolute readon nature of snapshots that allow you to have better confidence that if someone goof something up or something happen such as ransomware there's a point a readon point by which we can roll back to now ZFS snapshots are a function of the ZFS file system but we're going to be doing this demo in trass because it makes managing ZFS snapshots easier I will also note for those of you that want to know the really interesting details for how these snapshots work there's a great talk given at FreeBSD can 2019 that you'll find linked down below it does a great job of covering the details such as how do you figure out which blocks to delete when you take a snapshot out of the middle because that's completely supported and of course that comes with a lot of challenges and that's a great video to watch if you're looking for that as deep dive I'm going to be covering in this video the more functional side of how to set them up how they work how they present in Windows and how to see things shown the command line so let's get started [Music] are you an individual or forward-thinking company looking for expert assistance with network engineering storage or virtualization projects perhaps you're an internal it team seeking help to proactively manage monitor or secure your systems we offer comprehensive Consulting Services tailored to meet your specific project needs whether you require fully managed or co-managed IT services our experienced team is ready ready to step in and help we specialize in supporting businesses that need it Administration or it team seeking an extra layer of support to enhance their operations to learn more about any of our services head over to our website and fill out the hire us form at Lauren systems.com let us start crafting the perfect it solution for you if you want to show some extra love for our Channel check out our swag store and affiliate links down below that will lead you to discounts and deals for products and services we discuss on this channel with the ad read out of the way let's get you back to the content that you really came here for I want to start by discussing exactly how snapshots work at a very functional level inside of ZFS they are readon points and time so there is a time and date by which you took the snapshot and it is a frozen point in time at which that data lives it is readon as presented to an NFS share or an SMB share so I will show later in the video how this can be accessed and how you can see the previous versions but the end users who are attaching to that share just get a readon version of the data that data that snapshot is a great point in reference to be able to roll back to in case you ever need to come back to that data it's not a backup cuz it's all living within the pool more specifically it's living attached to either the data set or zal by which a snapshot was taken so if we take a snapshot and we delete that data set that the snapshot was based off of all subsequent snapshots attached to that data set go away and the same thing applies eval now absolutely the most common question that kids ask is how much space does a snapshot take and it takes the difference in blocks change from one snapshot to the other this is a very important concept that I did not say differences in files changed or differences in renamed files or way you shuffled a directory structure around it is very specifically both for zal and data sets based on the Block level changes the reason this is important and to understand what block level changes are is as we know data is stored simply as ones and zeros the Block Level storage that all the mechanics of ZFS that go into it is just ones and zeros at the end of the day those ones and zeros are then presented to you as a file system with folders and data and structure so it's more human readable not to say that binary is not readable by some humans but what we have here is if we rename a file and we do a snapshot we can roll back to the renamed version of the file but the differ in data stored is only the few kilobits of block changes for a file having a different name even more specifically if you have a large say one terabyte file and you change 100 megabytes of a 1 terabyte file that snapshot will not be the entirety of that one terab file it will only be the 100 megabytes change this is actually really handy with snapshots because if someone does something and I'll use ransomware as a very popular example here in 2024 and they rename a bunch of files and I say rename because ransomware does two things it renames and encrypts but it doesn't encrypt the entirety of the file it only encrypts enough so the file is corrupted so the difference from a ransomware attack would be only the blocks that are changed on the file which is going to be the name of the file and then the blocks that were corrupted well that's provided the ransomware actually renames it if it doesn't it's then still does the same thing so when a ransomware attack happens if it's only corrupting a certain block or encrypting that certain pieces of the block the difference change is not the entirety of whatever was stored it is only those differential blocks this is actually why ZFS works so well for rapid recovery from things because if you have a snapshot prior to a piece of data being destroyed you can snapshot right back to it and this all happens really fast I'll also note that databases are also something that you can take snapshots of although that can cause different issues because databases if you don't know where the transaction was in that place but conceptually if you are snapshotting something with a database it is only going to be those differential blocks from the different blocks that were written to that database so it's always looking at the Block Level which means it doesn't take up that much space depending on how many blocks have changed now as far as taking snapshots you can have your active data then your snapshot one snapshot 2 snapshot 3 which of course you'd want to name them all probably the time at which they are taken I'll show inet that this is done pretty much automatically it's really easy to follow but what happens if you want one of these back well it's easy enough to just roll back to a point in time but snapshots once again because they work at the Block Level will have the disadvantage so to speak of being able to roll back a single file they'll roll back everything that was within that data set or zal to that point in time but you actually have the option if you want to 4K snapshot so you can actually restore it elsewhere I'll also show you how to make them visible both for Linux and for a Windows share where they're going to present as a volume Shadow copy so you can easily let users self-managed ENT to be able to go back and find previous versions that are file I'll go over all that set up inside of trass one other thing I want to mention is what happens if you delete a snapshot out of the middle so you took one in snapshot one and then you have snapshot two which you deleted and snapshot three will it allow you to do that or does it need the whole chain technically it needs the whole chain but as far as how it's presented to you yes you can delete the snapshots out of the middle as well so if you took some extra manual snapshots and then have a bunch of automatic ones around it you want to later to Del those it sorts this out automatically and if you're thinking wow that's some complicated stuff going on that makes it happen go back to that video I referenced earlier at BSD can talking about the complexities of actually making that work all right this system is running Dragonfish 24.4 released candidate one here in April 2024 so it should be the same as the full release cuz snapshots haven't really changed much throughout the last few versions let's go over here to our data sets we have a test data set and we have a fully setup YouTube snapshot demo with which we'll get to shortly but I want to talk about how we got there and cover some of the basics so we have this test data set with 7.9 5 gigs of data and if we look to manage snapshots it doesn't have any right now so let's go ahead and start making them go to data set we're going to click on the test data set here and we're going to go ahead and create a snapshot we'll leave it at its default name of manual 2024 415 1148 it's the time date but you can choose a naming schema if you want and and recursive means also snapshot sub data set we're get to Next in the demo here so first let's go ahead and save that snapshot and if we now manage the snapshots we see the one and we see there is not available under space used that's because there's not any space taken because we haven't made any changes to the file system and retention is will not be deleted automatically this matters more for when you're setting up the automated snapshots which we'll show in a moment and when you set those up you set a retention and it's the snapshot task under data protection that actually facilitates deleting them and you can override if it's one that will automatically be deleted by putting it on hold now let's go here to the data set and you see that it's under Dozer and then test we're just going to do the command line and show you what the files look like in there so we're going to go here to Mount Dozer test LS and there's some data matter of fact du- sh some more data we have rounds up to the eight gigs in here let's go ahead and Purge that 8 gigs so if we rm-rf some more data now we're going to back out of this directory because we've now deleted it just show that it's gone and we'll go back over to our trass and now we're going to click on manage snapshots and now that it refreshed it says we've now referenced the 7.95 gigs because well we deleted all the data let's go back over here to our data set and our data set still takes up 7 .95 gigs matter of fact something important to note is that this data set takes up not just the storage of data that is active but also all of the snapshots and whatever the snapshots are holding until you delete them so it's cumulative the snapshots plus any other data in there and the snapshots are referencing that differential so let's go ahead and manage the snapshots again and let's just roll it back we're not going to do a new clone or anything like that we're just going to go ahead no safety check just roll it back to the way it was cuz we want all that data back back in there hit close and we see it's still using the 7.95 gigs we click to manage snapshots and this refreshes to go back to not available because there's no data used and if we go back into our test all the data is back let's go in here and prune out some of the data so there's a few files in here and I have something called pre-roll ad so let's go ahead and rm-rf pre-roll AD Maybe uh rm-rf whatever is in 2023 as well and if we look there's only three and a half gigs of data in there now now the problem is what if I wanted some of that data back well that's where we're going to go in here and this will refresh in a moment refresh this and it's only using 4.48 gigs say that's how much data we pruned out reference was the original 7.9 used that's how much it's holding on to if we roll back but what if if I may changed other files I want to roll it back but I want to get some of those files out well we have two options the easy one here is going to be clone a new data set and we can just say clone it here and it'll create an entirely new data set so we'll go ahead and do that then we'll go to the data sets and here's our test data set and here is that test manual clone that we have please note if you click on this data set it shows the total amount of data used versus this data set it has an option to promote but shows no data this is essentially a linked clone of this one so it's actually not taking up any data because it's referencing this one here but it will take up data if we promote it because then it becomes a fullon data set so it's like a temporary data set if you will where we can go inside of here and pull out files we do an LS and we have our test- manual we have all the data in there some more data and there it is and I can simply copy that over and we'll just copy it right to the root of test so it's in another location all right we've copied that data over we're going to back out of that directory and then we're going to ahead and delete this because I don't need it anymore so hit copy to clipboard and delete I found the files I wanted I'm going to clean up the extra data set and it goes away and the snapshots are still there but now we have more data cuz we copied data from there and then put it in a different place now by doing this we have some differentials from the different snapshots that we have from the different times so it's actually going to end up being a little bit more data because we have to now keep track of the changes and the extra data that I copied over into this test folder and then we can do the same thing again we can create more snapshots and the process continues now let's talk about how to make them visible if I didn't want to have to go through and have these so let's go ahead and create another snapshot and it'll just be called manual this actually we'll call this another Dash manual snapshot we'll go ahead and hit save and we're going to go here to this test data set we're going to go edit Advanced options and we want to change the settings here from snapshot invisible which is the default to snapshot visible go ahead and hit save and we're going to go back in here to mount Dozer test and we have a new directory we can see called ZFS and if we go to our ZFS we see snapshots and there they are there's our manual snapshot and there's another manual snapshot so we go into the manual snapshot there is our some more data and there's some of these different files are in there now I mentioned them being readon please note I'm logged in as root and it tells me it's read only even when you're root this can only be controlled via the ZFS commands so you're not able to actually do any data manipulation these are as I stated read only so I can go through and manually see them now let's talk about how this is handled on a Windows share with volume Shadow copies because not everyone's going to be doing this from the command line or has the time to deal with a lot of end users using it and this is a really important feature so we have our YouTube snapshot demo set up you'll see that we have 60 snapshots and the reason is because we have an automated task and we'll cover those in a moment but this automated task basically says do a snapshot every single minute and of course doing it every minute means there's lots of versions to roll back to this is probably more than you need but it also makes it easier for me to do demo here so let's go look at the data set and we see that it has normal share permissions for built-in users and administrators and it has an attached SM rooll so if we look at the SMB share everything's pretty standard but I want to highlight this we're going to go ahead and edit and we're going to go show Advanced options and we're using the default share parameters in the default share parameters is enable Shadow copies this presents those snapshots as a volume Shadow copy meaning it's easy for users to have a readon available version that they can previously roll back to inside of Windows something else I want to note is if you create the share prior to creating these snapshots it won't know to share them to solve that problem we're going to go here to system settings and services and you just simply start and stop the SMB service and when the SMB service starts and it sees that there's snapshots available for one of the shares it automatically converts them into Shadow copies it does this for all snapshots going forward as long as there's at least one in there it enables that feature now let's show what that actually looks like so we have our test data here and we have our pre-post assets just a few things I threw in here just copied them out of my YouTube directory and what if we went and deleted some of the things in here or even all the things in here and hit shift delete delete these 11 items all right if we go here now and uh that's obviously an empty folder but let's go to restore previous versions and I've actually manipulated these a couple times I can look at all the different previous versions I had uh here's one from just a little bit ago and let's open it and we can see all the files are there or we could just go ahead and hit restore yeah let's just restore that cuz we really didn't want to delete all of those and we go back in there and they're back now this doesn't just apply to folders of course this applies to things like the text files themselves so let's edit this file this is some random test wordss this is some more random test words and we'll just paste this all the way down so you now change and manipulate this file I just hit save and we'll close it and if we want to go to properties previous versions and uh let's go ahead and restore that previous version yes I'd like to restore it hit okay and now it's back to the way it was something I will note again that if we wanted to look in one of these folders right here such as pre-post asset and let's go to the properties previous versions and let's open one of those previous versions of this there's no way for me to delete any of these I can only copy them there's no way for the window share because it's all controlled by the ZFS commands on the chass itself uh for them to be able to delete this so in the event of something like a ransomware attack none of the Shadow copies can be deleted there's no way to remove them because they're only emulated Shadow copies that are actually based on ZFS snapshots so your data is protected but let's talk about the Nesta data set here because if we go here and we right click and do properties and previous versions there are none and let's explain what a Nesta data set is so we have our YouTube snapshot demo and we have the Nesta data set the nested data set is a way you can put a data set under another data set this allows you to have different permissions parameters and even different share options or in this case specifically the data protection options as far as the snapshots so even though this snapshot right here there are 60 of them and there's a task because in this particular task if we edit it the recursive box is not checked if we check the recursive box it'll automatically also snapshot that particular data set or we can go here to data protection and we can add a snapshot task that is different because maybe we have different retention policies for different data sets and that allows us to have a retention policy that would be for specifically this that's a data set that can be different maybe we need to keep the data longer we want the snapshot lifetime to be longer Etc that is all features of a data set and we can have those different ones in there and the same thing if we're creating things manually so if we go back over here to our data sets if we were doing a manual snapshot doing recursive means grab all the subsequent ones underneath here as well so that's what recursive means just like it sounds it recursively does all the data sets that are underneath here to follow that policy now one thing I want to note is if we do that and let's just go ahead and edit this task real quick here and we go to our data protection snapshot task and we're just going to make it recursive by doing so we'll give it a minute here because right now if we look at our data sets and that one runs every minute there's currently a total of zero so I'm just going to fast forward here and wait a couple minutes all right now we've jumped ahead a couple minutes so we have a couple snapshots here for that new Nesta data set so it works and has the same retention policy as the other data sets do based on that setup but let's go back over to our Windows system and it still shows no previous versions available well that's actually because there's not any differential that we have here so let's go ahead and delete something out of here restore previous versions and there we go now they're showing up now the last piece I want to cover is actually creating these tasks it's pretty simple uh but I'll cover it really quickly here we pick a data set and I don't recommend doing this and I've seen people do this well they'll just pick the root of the pool see recursive and suddenly have snapshots of everything this kind of creates a mess and I don't think you want to deal with that I recommend having a plan where you take each one of the different data sets you have and set a retention policy that makes rational sense to you when doing this maybe this Nessa data set needs 2 weeks of Lifetime and uh that's fine and we can even say that we'll call this one two- weeks Das Auto and that way the naming scheme these are the two week ones that we want done daily and we can exclude if we need certain filters here you can be specific about how you exclude them if you're doing it recursively but if we were going to set a retention policy for this specific data set this specific Lifetime and you can choose the units of hour day week month year maybe you have a really long retention policy but of note when you have these the important part is that you have a process by which you know where they are because if you decided to keep a year of them you could run out of space please remember all snapshots are cumulative to that data set so if you have only x amount of storage and you exceed that amount of storage in differential changes over a year well you will run out of storage on there so as much as it may be wonderful to keep a year worth of snapshots it may not be practical depending on your storage needs just something to keep in mind here that's also why it's important to have of the Nessa data sets because some things you can set a really long retention policy cuz maybe they don't change often and they're just really archival so having a long retention policy is not a problem and it can minimize some risk versus if you have a really frequently used folder that may not work as well so you may want to assign that to a shorter lifetime once again it's going to vary greatly based on your use case if you're still noodling around about that nested data set and about different retention policies this this is actually some projects we've completed for different government agencies and yes the government uses a lot more trest than you may realize that store this data but then some of the data is only archived slowly like every 30 days they add there so they have a really long retention policy and the fear would be what if someone deleted something we wouldn't notice for so much time so they can have archival settings of years because essentially once the data is manipulated into there it's static of course there's lot ways you could set this data to read only after they have a lot of different policies around it but that's what having all these different data sets nested underneath each other with all different snapshot and retention policies that's what it's for Is For Real complicated business environments with sometimes some rather unique use cases now I didn't do this demo with Zales but I did mention beginning this works with Zales the most common use case is going to be if you have a zal that's attached to a server that you're running inside of trass as a VM this is a way you could easily manipulate the snapshots or even when you do a clone of a virtual machine that is how that's handled the other use case of course is if if you have ice scuzzy attached which is block device shared out to another system now the challenge becomes when you roll those back because ZFS doesn't necessarily have any idea what's inside there it's just manipulating the blocks does the OS that you've attached this to understand the differences when you roll it back or does it have some file system marker it's keeping track of uh I bring that up because there are times and I've had a lot of people have challenges specifically like with hyperv when they've attached hyperv to an ice gzy hyperv seems to not like when you suddenly roll back to a previous state and is expecting something else and now you've presented it that's just a few words for warning but this works much the same as far as ZFS concerned but you do have to take into consideration whatever you're attaching it to will it be fine with the rollback just something to think about and some food for thought like And subscribe to see more content from this channel head over to lawren systems.com to connect with me on whatever socials I'll be on when you go there also you'll find the ability to subscribe to my newsletter which I'm doing a monthly newsletter now so if you're interested in all the videos and some of the news articles that I share in there go ahead and sign up for that and find me over in my forums forums. laoren systems.com to have a more in-depth discussion about this and other topics I've talked about on the channel and thanks [Music]

Channel: Lawrence Systems

Views: 18,162

Keywords: LawrenceSystems

Id: 3J-27jm8cU8

Length: 25min 3sec (1503 seconds)

Published: Mon Apr 15 2024