Easy rsync Backup with tar and cron (daily, weekly, monthly)

hey everyone this is tony teaches tech i'm tony and in this video i'm going to show you how to do a daily weekly and monthly backup of a directory on your linux operating system now specifically for me in this video i'm going to be backing up my wordpress website directory but like i said this could be any directory on your server we're going to be writing a script that utilizes the rsync command the tar command and the find command and we're going to call that script with a cron job so let's go ahead and get on into the tutorial here okay so i'm logged in to two servers here via ssh this one on the left is the source server so in here we're going to be backing up my wordpress website like i said at var www.html and then over here we are having the destination server and there's there's really not much going on here um but first i guess the cut the way that i want to set this whole video up is to go over the individual commands that i mentioned and give you a fundamental understanding of how they work and then we'll write them into a script like i said and then we'll call that with the cron job which automates this whole process and is something really good because you never have to worry about dealing with your backups the cron job will take care of that for you okay so let's start out with the tar commands because the tar command basically creates an archive file of this directory and all of its contents so if we just test it out here on the command line we can type tar z c f and these are just three flags um i think it's gzip compress into a file and then the file that we want to create we'll just call it for now backup dot tar.gz and um i'm gonna i'm gonna go right into telling you about this dash capital c flag which is basically we're going to change the directory to var www okay so the capital c is change directory into this directory and then what we want to contain in the archive is the html directory so when we go ahead and execute that that's going to take all these files and put it into a file called backup.tar.gz now if in the future for example if we ever want to utilize this backup we would execute the tar command again but this time with um the x command or the x flag for extract and x z v f this v is verbose um and then we're going to give it the name of the file that we want to extract so that's backup.tar.gz and that's going to extract all of those files into this current directory and then we'll see the html directory and if we go into that directory we will see all of our website files in here okay so that's basically how the um tar command works so what i'm going to do is get rid of all these files because that was just like a a quick demonstration here um but what we want to do because we're going to be executing this command on a daily basis we want to give each backup file a unique name so we want to end up here with a daily backup where each backup has a timestamp in the file name of what day it was backed up okay so in order to do that we can use the date command to give that file name a specific date so let's try that again instead of i'll just use up arrow key here to go back to that previous command instead of calling our backup simply backup.tar.gz we can say backup dash and then we'll give it a dollar sign into parentheses so this is basically just going to execute a command between these two parentheses we're going to execute the date command and this this might get a little confusing for you but if you need to copy and paste these verbatim i'll have them linked down below in the description but this is just the format of the the string that we're working with so um percent y for the year percent m for the month and percent d for the day so that's going to get the current day in the format of year month day so if we execute that this time it'll create the same file but the file name will be backup dash 2020 one it's 2021 now um excuse me 2021 01 for january and 23 for the 23rd of january okay so that looks good um let's remove that again because we're we're just going through the the motions here um the only other thing that i want to say here is that it's probably a good idea to make it a little bit organized so let's make a folder here called backups and then in the backups directory we will make another folder in here called daily and then we'll make another folder in here called weekly and then we'll make another folder in here called monthly so again the goal here is to have a series of daily backups maybe going back maybe a week or so a series of weekly backups you can pick a number maybe 10 weeks will go back in a series of monthly backups so at most we'll keep maybe a year's worth 12 monthly backups so the reason we want to do this is because we don't actually want um to keep like 365 days worth of backups we don't want 365 daily backups we only care really about the seven most recent and that's configurable you can change that to whatever you want but that's the way i like to set it up for myself so anyway let's modify our command just slightly so that our daily backup ends in the daily directory so we'll hit the up arrow key to go back up there and we will do instead of this we'll just add to the beginning of the path here and i'm going to use this tilde character to reference the the home directory slash backup slash daily okay so now when we execute that command our backup file instead of ending up in the current directory it's not in there it'll end up in the backups daily directory okay so that looks pretty good um let's let me show you next this command uh it's the find command you might be familiar with it so the find command allows you to find typically in a use case find files given a name or a certain pattern but what we want to use the find command is to delete older versions of files that we don't care about so like i was saying we only want to keep in this case at most seven daily backups so with the find command you could do something like this so let's find inside the backups directory inside the daily directory any file that has been created and has been created is the dash m time command over seven days before today that's the plus seven and then what we want to do with those files is delete them so if we execute this now we don't have any uh files in that directory that are older than seven days so nothing actually happens but i just want to prove that this works because um this if if we change this plus seven to minus seven that just means that any files newer than seven days if that makes sense so like what it's going to do is look out seven days from now and anything that's been created before seven days is going to get deleted so what this is actually going to do is delete that file because it was still created today so if we execute that and we do an ls of the backups daily directory that file is gone so i don't have any um i don't have files older than seven days to prove this to but just i guess take my word for it that this will delete files that have been uh created over seven days ago and that way we will only have at most seven daily backups in this directory okay so let's keep that in mind and then uh actually what yeah we'll keep that in mind so we did the tar command to create the archive we're going to delete old versions of the daily backups that are greater than seven days old with the find command and then the next thing to do is to pretty much copy everything in the daily directory over to from this source server off of the server to the destination server here and we can do that with the rsync command so let me make one more backup here just so we have something to work with and so we have our backup in here and now let's let's play around with the rsync command so we can type in rsync a for archive and then similarly dash dash delete is going to get rid of if if if this file has been deleted in the source it'll also be deleted in the destination so eight days from now when this file's deleted from the source it'll also get deleted from the destination so what that does is actually makes a mirror image between your source server and your destination server they'll be exactly the same all the time okay so rsync dash a dash dash delete what are we backing up we're backing up i'm going to give it the full path here from the root directory we're going to back up the entire backups folder so all of our daily backups all of our weekly backups all of our monthly backups everything under there and that is our source and then the destination is this server so rsync allows you to log in to that server so we're going to give it the username of this server over here at this ip address you can see up here so 161.35.143.122 and then you type colon and then the relative path to where you want that to end up on the destination server so i'm going to use the tilde again which references the home directory on the the destination server so slash backups now before we execute this we want to make sure that the backups folder exists on this side so let's go over here and make the backups folder okay so now we have a backups folder and obviously this is empty by na right now but when we execute this command and give it our credentials we will see those files transfer over so let's go ahead and do that so hit enter and it's asking us for the password for this and i'm going to copy it's asking us specifically for the password for the destination server so i'm going to copy and paste that here you won't see anything show up on the screen i'm going to hit enter and that's going to do its thing now over here on the destination server let's verify that it actually happened we can do ls backups again and we'll see that we have our daily monthly weekly folder and if you look in the daily folder we will see that we have our backup file in here so really good that's that's uh basically the gist of what we're trying to do here at this point we can there's one more thing we want to do we we can't because this is an automated process we can't be typing our password in every time so there's a way that we can um pretty much allow for a passwordless login to this remote server every time and we can do that with our ssh key so i have other videos on this topic so i'll just go through it briefly but what we're going to do is basically look if you don't in your ssh in your home directory excuse me there's a directory in here called dot ssh and in here if you don't have an id rsa and idrsa.pub file which is your your private key and your public key for ssh you'll have to create one and you can do that with ssh you can do that with ssh dash keygen t for type rsa and then just go through the process of generating that and that will give you the idrsa private key and public key but anyway since i have that i don't have to do that what we're going to do specifically is look at the my ssh idrc my public key okay and all we have to do is copy the contents of my public key okay we'll get rid of there and come over here to the destination server and inside that same directory inside the ssh directory you should have everybody i'm pretty sure should have a file in here called authorized keys and we're going to put the public key of the source server in the authorized key file on the destination server so that way when this destination server is presented with the public key of the source server it's going to be like oh i already know who you are so i don't need your password you're allowed to log in so let's test that out let's see if we can r sync this file back over here without typing a password and this is this is not necessary at all but i'm going to go into the backups directory and just remove the the daily folder um just so we can see something happening here but we're actually going to be looking for two things we're going to be looking for the fact that this is not going to ask us for a password and the fact that it actually copies over so let's go ahead and execute this command it doesn't ask us for a password because we set up the the keying and then we can ls over here to see what's in here we got the daily directory and inside the daily directory we have that backup file cool so let's put it in the script let's make a script called backup oops backup dot sh and what we're basically going to do is take everything that we just wrote and put it into a script so we have our tar command we're going to take the time stamped file we're going to make a timestamp file oops i'm going to make a timestamp file for today with the contents of the html directory at var www we're going to delete anything that's more than seven days old and then we're going to take we're going to call the rsync command and backup everything from the backup directory on this server to the backup directory on the destination server and that's about it so um let's again just for the sake of demonstration let's let's remove everything in this backups directory on the destination server there's nothing in here right now but if we call the backup the backup script that we just wrote it will execute all those commands in sequence that we just created so it should create another tar file and actually we should remove this backups directory too okay so um oh no that was that was silly we need to keep that backups directory so let's make that again because it needs to exist for it to show up since so make directory backups and then inside backups let's make a daily make a weekly and make a monthly okay sorry about that so i just want to i just want to start from like a clean slate so we know um we see this happening so let's go ahead and call that script sh for the shell um and then backup.sh okay so let's execute that and it's something went wrong back up oh i right i this isn't plural this is a singular so let me um let me modify the script it's always hard to do these videos live because um you always you always screw something up so it's back ups with an s and back ups with an s and back ups with an s same thing over here okay so that should work now let's try to execute that script one more time and we don't see any output that looks good it's it's doing its thing so let's check on the source in the backups directory um in the daily directory we have created that file and then in the destination directory we have daily monthly weekly and then in the daily directory we have the backup so that is the general way that the script is going to work obviously you can modify it to your needs change the paths change the the time stamps all that stuff but that is the fundamental way about how this works for daily backups now if you wanted to do this for weekly backups we can make a new script called backup weekly something like that and we can do something very similar so uh i'll just copy and paste this here you guys should know at this point what we're doing so we're gonna into the weekly directory make another time stamped file and the source is going to be the html directory at var www and then we're going to get rid of anything in the weekly directory with the find command that has been created over 31 days ago so i and again this is configurable so uh all right yeah 31 days ago which would be a month um so that would be kind of what your weekly script would look like and then let's just do for completeness like a monthly script and that would look something like this so again into the monthly directory create a time stamped file archive using the tar command of the contents of the var www.html directory and then we're going to get rid of anything in there that's over 365 days old or a year old so we're going to keep a year's worth of monthly backups okay and now the final piece to this whole puzzle is to automate this with a cron job we can do that with uh typing cron tab dash e to edit and in here i might already have something yes i do so i am calling pretty much what we typed on the command line um except i need to add an s here to backups so we're giving it the full path to our script so slash root slash backups slash backups do i have that path rate let's see we have a backup.sh file in the root directory so now i have that slightly off so uh to do so it's just root slash root slash backup dot sh so what we typed on the command line pretty much we're gonna call this script and then if you're not familiar with cron jobs basically this syntax right here is saying every day of the month you can kind of see it here every day of the month every month of the year every day of the week at 12 o'clock midnight or i guess in this case at 12 15 midnight the hour and the minute so 0 15 at 12 15 midnight every single day we're gonna call this script so that's going to generate that daily backup the weekly backup well in this case is going to generate the daily backup but in order to have a weekly script we would do something similar uh like let's say i don't know at 12 30 um just once a week we want to do this so we're going to call the um the weekly script just once a week so it's going to be star star and then the day of the week it's just like mondays one tuesdays tues something like that so just on let's say monday is number one here and then we're going to call root and then back up dash weekly and then finally for the monthly backup we'll say maybe at like 12 45 a.m the day of the month let's say the first of the month we'll back up and we'll call the root backup monthly script month okay so um and i might have spelled that wrong back here monthly backup weekly m-o-n-t no i got it right okay so basically yeah let's review this so we have three different cron jobs this one's gonna call on a daily basis at 12 15 a.m the backup.sh script this one is going to call the first day of the week which it could i could be wrong but let's just say it's monday at 12 30 in the morning uh the backup weekly script and then backup monthly script is going to be called the first day of the month at 12 45 a.m and that's going to execute and that's it that's really how how easy it is to create a custom backup solution for whatever directory you want on your linux operating system by this point you should know how to use the tar command the find command and the arsync command and how to do that password this login if you have any questions about any of this including the cron jobs let me know in the comments below i'll do my best to help you out and and i do have a blog post like i said with all of these commands in it so um also check that out as well if you got any value out of this video definitely consider subscribing to this channel for more videos like this from me in the future and if you do i'll see you in the next [Music] one you

Channel: Tony Teaches Tech

Views: 5,351

Keywords: rsync cron, linux backup rsync, website backup, how to backup wordpress website, how to backup a website, rsync tar, rsync tar backup, backup with rsync and tar, backup retention daily weekly monthly, linux backup script daily weekly monthly, backup daily weekly monthly, rsync backup, rsync incremental backup, rsync example, rsync linux, rsync command, rsync tutorial, rsync backup script

Id: z35ZPELo5_Y

Length: 22min 14sec (1334 seconds)

Published: Mon Mar 29 2021