How to backup a docker container | Docker Volumes Backups with Ease: A Comprehensive Guide

hey everybody welcome back to the channel Tech with Marco I'm Marco I'm a software developer and the devops engineer and as you know I'm working a lot with Docker and working with Docker implicates storing your data somewhere and for that mostly I use Docker volumes and with Docker volumes it is easy to store data and manage data within Docker containers but there is one big question when it comes to backing up your data So currently I have a lot of Docker containers I use in my daily setup with servers and I recently came across the question how do I manage my backups with all the docker volumes I already have and there I researched a lot about how to set up a nice backup strategy am I now preferred backup strategy that's what I want to share with you in this video so stay tuned and follow me along so first of all let's head over to GitHub and I will show you what a project I came across this is a project by often and it's called Docker volume backup and what it makes so special for me is that it's just another Docker container which I can add to my Docker compose setup and this Docker container offers a lot of options how to do backups where to store them and what to do after the backup has happened and it gives you a lot of notification options encryption options and other configuration options what I really like about the tool is that it's kind of straightforward to set up and it offers a lot of Integrations with different storage providers for example you can use AWS S3 file based Min IO which are kind of compatible with S3 you can use any web DAV storage service you can use use SSH for other remote servers you can use the Azure blob storage which is the bundle of Microsoft for S3 and you can also backup locally to yeah I don't know for example your just local server and then use some custom script to copy the backup and what a real killer feature for me is the gpg integration so I can encrypt automatically my backups and could even store them in a public accessible server and no one can use them because they don't have the gpg key to decrypt the encrypted backup let's head over to my IDE and I'll show you what the setup looks like in a Docker compose file so I'm now heading over to the docker compose file and as you can see we have the first two Services there are for example Docker serves which is called app tank Kuma which is by the way a really nice monitoring tool and first of all we can see it's just easy configured here we have an image we have a container name we have ports and we have volume where app10 Kuma is storing its monitoring data and now we have your label which is related to the backup service I'll explain that one later and I have also a mongodatabase here for example just straightforward image container name Port mapping and also another volume here as you can see this is the mongodata volume the bottom of the docker compose file I declared these volumes as Docker volume so they're persistent and just for showcasing here we have the Express which is just the Local web browser viewer for your database to demonstrate what data is in there and yeah I Collapse these three services again I come back to them later so let's head over to the backup configuration container here so we have the image which is called often Docker volume backup I'm using here the version two I gave that a container name which is just backup and restart as always so now we can mount an environment file and basically if you have a really simple setup you can Define every configuration option in that environment file I name my one here backup.nf and in that backup show that there I have configured common configuration options between the two other Docker services like Uptown Kuma and mongodb and now there is a crucial part coming in here we have to map certain volumes so first of all we have a backup configuration folder here and this backup configuration folder contains two different configuration environment files and those two configuration environment files they relate to one of these two Docker Services one is for the mongodb and one is for the apptime kuma service and we're mounting that one here into the folder Etc Docker volume backup slash conf D this is just a mechanism of the docker volume backup container to have multiple backup configurations for different Docker containers then we have to mount you don't have to but it's really not it's a really nice feature you can mount the docker socket into your backup container and what that does with the mount Docker volume socket here you can stop containers during backup times and I would really recommend doing that because this prevents the data corruption and you better have no corrupted backups of Docker volumes therefore I really recommend that and now we have the crucial part here so I'm mounting the data volume which I created here at the bottom for the DB and I'm mounting that into the directory backup slash data backup and the option here is read only so the backup container is not able to write into the volumes you read data from there and this part of the directory here you can name that as you want so I chose your mongodata backup so I know what the backup is about and the same is happening here for the uptime Google volume the mounting the volume here into backup app time Uma backup that's the directory path and now we can have a look at the dump environment file so I'm opening that here in a second tab here so in order to have multiple backups configured for different containers we have to declare a backup source for the backup strategy here and the backup sources backup slash and mongodata backup and that one has to match with the path which we have in the backup volume mounts here and this is data backup and this one also refers to the mongodata backup so this environment file here is the configuration for this mongodata backup strategy and now we can head over and create a different a greater backup name now we can create a backup file name here and this one is my daily data backup and then the year variable monthly variable daily variable that time is coming the hour the minute and the second and this one has the ending.archive and this one is a feature here which is called backup pruning prefix this is the beginning of the part up here backup file name and the backup retention days is seven days so I'm keeping the backup for seven days and after seven days the latest one is deleted and for this backup strategy I only want to delete backups which are starting with the name Bay daily data backup therefore I have to define the backup pruning prefix and that was a special part here the exact label which I then here and as you can see we have some labels here configured at Docker containers and at the container we have the label Docker warriorbackup.archive minus pre and what that does before the backup part of the backup container starting we are executing a share command here and what I do here is jumping all the database here into the TMP slash jumps slash mongodb.archivedirectory into that file and as you can see here in the volume rounds we have the mongodata volume mounted here into the TMP slash Demps directory so that means before the backup container is starting to backup we dump we are dumping our database completely into this TMP slash dumps directory and as this one lays in here the docker backup container and not any kind of different database files so this is a really recommended strategy for backing up databases so I really would recommend you to not store the whole volume for example you could delete everything here and just not the mongodata volume into the whole file system of the container and delete all the labels here so that means we just really want to have the database files or the database entries and databases and stuff like that so this is the recommended strategy for backing up databases you could also refer that to mySQL and MRI ADB and there are a bunch of different databases out there and now we have another backup strategy implemented so I'm collapsing here and going to the uptime kuma which is as I mentioned the nice monitoring tool and there we don't have these labels which are called Docker volume backup archive pre we have just a different one here which is called Docker volume backup stop during backup time and I'm opening the Mongol the uptime Google environment file here next to our Docker compose file and there we can see we still have to configure the backup sources directory this has to match as I said the volume Mount here and then we are giving that a different name a different backup pruning prefix the same retention time and I'm creating here a backup stop container label uptime Kumar minus service and I'm referring to this label in the label configuration here and that means when the docker container uptime Kuma has this label Uptown Kuma minus service then this complete Docker container is stopped before the backup is taken and at the bottom we also have a backup Chrome expression so you can enter there any current expression one there are a million of possibilities and now we can have a look how that works oh I forgot one thing so let's have a look at the backup.enth configuration so this is the general configuration which is valid for the whole Docker volume backup container as I said in the beginning we can have different storage packets as Amazon S3 or any webdav server and for demonstration purpose here I use the webdav server and this is a managed next cloud in the next Cloud I created a folder which is called Docker minus volume minus backup and this one is referred to the web DAV path then I have a username I have password to log in now there's also another crucial setting here I'm encrypting my backups with gpg here you can just omit that like delete that out of the configuration if you don't want to backup to be encrypted but I also recommend encrypting backups so I chose here the passphrase hello test I recommend using a better one here and as I just put that one in the general configuration here both backup strategies are using this password I could also use different passwords for the different backup strategies and that means I just have to enter the gpg phrase into this passwords here for example into this environment file and I could also use a different password for the other backup strategy but I don't want that I'm using the same backup strategy I'm using the same password here and I also configured a notification a URL here and you can use emails like Discord there are a few of them there are linked in the description of the backup volume Docker container and I set the notification level to info otherwise if you don't set that to info you're only getting errors notified to your notification URL here and now I can start our Docker compose right here I quickly show you my empty next Cloud here so I'm opening my next Cloud here and when I'm heading over to files we can see that we have a folder here which is called Docker volume backup and if I go in there we can see it's empty and now we can start our Services here our Docker compost deck and as you can see we have now the backup container the container the Express container and with the Upton Kuma container I'm just expanding that here so let's head over to the Express for example localhost.8081 and then we have a database I can just enter a test database here and now we have a test database in our here uptime Kuma is also sorting so now we have the configuration here so I just created my admin account at uptime Kuma here at my local host machine now for example if I have to delete all my servers getting destroyed or whatever and I would lose this account for example or just monitoring data in here and as I don't want that to happen uh that's why I implemented my Docker backup strategy here now you can head over to the output of the docker uh backup file and as you can see we have here a pending chron.d entry with expression every three minutes and the configuration file mobile dot uh minus dump.environment there we can see that now the three minutes are over and the docker volume backup archive pre-command is running with top minus archive for the container and yeah we have created the file successfully now the often Docker volume backup container is encrypting the archive with my gpg key with my passphrase which is hello test and the finished running backup task is completed and then the next backup is already happening here so we can see it's stopping one container which has the label uptime Uber minus service and yeah that time Kuma service was shut down and then the backup was taken here and also encrypted with my passphrase and after that both of them were uploaded to my next cloud webdav back in storage and I also enabled the select notification URLs here so as you can see we have our app open backup and now you can see that the local output was level info and it's yeah created backup here and yeah I just got the notification that everything worked so 12 50 12 48 and uh 48 so every two minutes and now we're heading over to our next cloud storage here and if I am now reloading the page I would expect to see some files and yeah we can see we have the daily data backup archive.gpg and then we have another daily uptime Kumar backup and again another one just seconds ago so there are two minutes schedule as a running so everything worked perfectly and now you have encrypted backups of your data how awesome is that so that gives me one less headache to worry about and now one of the reasons why I chose the docker container often Docker volume backup instead of creating my own backup scripts is I can just include like three configuration files and one sidecar container in my Docker compose setup and they are easy as that I have a remote backend configured I can do Chrome Expressions I can shut down containers I can customize my commands in which I can Define how the backup in the container is working or if I just want to back up the whole volume and really easy as that I have my complete backup strategy implemented so after you've backed up all your Docker volumes you are now able to restore your data in the case something is happening to your server or your infrastructure and yeah I hope you really enjoyed the video about my Docker volume backup strategy and I really appreciate if you subscribe to my channel give the video a thumbs up and yeah just give me a comment what is your backup strategy and yeah catch you guys in the next one bye bye take care

Channel: Tech with Marco

Views: 7,347

Keywords: tech with marco, docker, backup with docker, docker volume backup, dataprotection, security, backup strategy, hosting with docker, aws s3, nextcloud webdav, docker-compose backups, docker volumes backup, how to implement docker backups

Id: qlo0Z1I1DD0

Length: 17min 23sec (1043 seconds)

Published: Mon Aug 14 2023