Migrate Docker Volumes from one Host to another // backup and restore

hey everyone this is christian and in this video we'll talk about darker volumes and how to migrate them from one host to another i show you where docker stores all the data the different types of storage and also how to copy any existing files from inside the containers and use them to create a backup file and use that to restore or migrate data and i often needed to do that when i was switching or migrating my containers from my old home server to a new one or when i was switching from my old vps hosting to a digital ocean and kubernetes so there are often situations where you store data in persistent volumes you need to migrate so let's talk about that and let's take a look at my docker setup and assume i want to migrate some existing containers to another host as you should already know docker containers themselves are immutable and they don't store any persistent data by default so when you just destroy a container and redeploy it all the changes you have made to the file system are gone and when you have an application running on one server you want to run on another server instead you simply can't destroy the existing container and start the same image with the exact same configuration like the environment variables and all the settings on the second server but of course there are situations where you need to store data persistently like on databases on web servers or file and media applications and mostly you will use local docker volumes to do that i said local because there are also other ways to store docker volumes for example with network protocols like nfs but for now we are just considering what most of you guys are probably using the local storage and that means the data is stored on the same host where the container is running and here there are two ways to do that i've shown you this in previous videos where we deployed containers with portena or in the docker cli for example and we were mainly using volumes or binds in case of vines it's very easy to migrate the data because it's just a mount point on the host machine and you of course can just a cd into that path access it and just copy the files away but how do you do that on a named docker volume where you don't have a specific path on the host machine you can access the files so let's take a look on my home server where i'm running a bunch of different containers and applications for example homer which is a nice home server dashboard that i've shown you in my last video and if you haven't watched that already then you should do that it's really really cool so when we take a look at the settings of this container you can see that there is a volume that is a binding so this binding is simply just a path on the host operating system that is stored in my personal home folder and i can just go into that folder on the host machine just access the files and save them migrate them to whatever location and change them whatever but i'm also running some other containers for example the bit one container which is not using a binding but instead a named volume you can see it here this container is using a named volume that is called bit warden and the path inside the container is stored in the data location in portena when we know click on that bitboard container it will reveal us the real storage location where docker stores all the volumes so you can see that all the docker volumes that are named volumes are stored in the var lib docker location under volumes and then the name of your container and then it depends on where the data is mounted inside the container how that directory is looking like you can also do that in the docker cli for example if you don't run portena and execute the docker ps you can see all containers that are running and to inspect any of their container settings you can use a command that is called docker inspect and then the name of the container you want to get the information from so for example these are all the settings and configuration of the bit one container and if we scroll up you should also see the mounts that we are using and you can also see that this is a local volume that is called bit warden and the source location is stored into this path here and when we cd into that path here and do an ls command you can see all the data inside the darker volume so you can also access it you can copy it away and this requires sometimes different permissions so sometimes you need to switch to the root user so otherwise those folders might be hidden and you might not be able to access those files here but when you are running this with the correct permission you can simply just access the files just like from inside the container and you can easily copy them away so this might be enough to access the files but i also want to show you how i back up restore and migrate the data because if you want to copy every single file to another location it might take some time depending on how big those files are for example in a large database this can take some time and i usually use a single one line command to create a table image file which will compress all the data inside this docker volume and this is a simple one line command you can easily find on the official docker documentation i've put your link to this in the description and there are there are some commands and instructions how you can simply up a container create a table image and restore a container backup as well but i've also simplified this a little bit and created a cheat sheet for you guys and you will find that on my personal github page under cheat sheets i will put your link to this in the description as well and then you can find a cheat sheet for all the different docker commands that i'm currently running and have documented if you know something else and you have some cool suggestions just feel free to um create a contribution or create a pull request and i will take a review on that and i also added a section for you where you can simply just use a single command to backup and restore those container volumes and i've modified the command of the official documentation a little bit so let's take a look at this and go over this step by step so this command will run a new container which might be surprising to back up some files here but this is how you can actually do this the rm parameters are there to remove any existing containers that might use the same name so that is usually used in any helper containers that are temporary and we also want to use a second parameter that is called volumes from and now we want to attach all the volumes that are used for a specific container and attach them to our helper container as well so you don't need to remember which volumes are mounted at which point of the container and so on you simply can just mount all the volumes from the container bit one for example and use them to back up your data i also want to add a second container and this is mounting the current location that i'm using in the terminal right now so this stands for the current location that's inside my personal home folder at this time here and i want to mount this inside the container and want to create a mount point that is called slash backup i also will use a different image than in the official docker documentation because i'm not using the ubuntu image i'm using the busybox image which is a more lightweight image which has all the necessary commands to debug maintain or backup containers so this is very useful and then i want to execute a command and create one single file you can easily transmit or transport to a second server and this will be much faster because it is just a single file that is compressed and i want to create this backup from the container path so now we need to look where the data is stored inside the container and this can be different depending on which application you are using so if we go back to portena you can see that this volume here for example stores or mounts the data inside the container at the location slash data this is the case for bit1 as i said it might be different for other applications and we want to backup everything that is stored inside this folder here so let's enter this command and you can see that's now copying all the files that are stored inside the container and it has created a new table archive on my host machine that is called backup.tar and depending on how strong the compression is this will save you a lot space and this will make the file transmission to another host so much faster so for example you can just use scp to copy this table image to a second host where you want to restore the data for example and this is a typical way how you can deal with backup and restore as i said if you want to restore data from the table image you can just go back to my cheat sheet and use the second command and you simply can just again modify some of the parameters here and restore the data from your towel image and you can simply just use it on a second host if you want to migrate the data or if you messed up the container or the volume for whatever reason and you need to recreate it and restore the data from that backup so i hope this helped you to understand how docker volumes work and how you can access the data inside these volumes and migrate them to another host or use that to backup and restore data but if you're really interested in managing regular backups for example managing incremental backups and save this to a cloud location for example i've created a separate video on that i will put your link in this in the description down below so this is a very cool video about duplicity and how you can use that to manage your darker backups so watch that next and as always thanks everybody for watching i see you in the next one bye

Channel: The Digital Life

Views: 6,207

Keywords: docker volumes, docker volumes explained, docker volumes tutorial, docker volumes vs bind mounts, docker volumes vs mounts, migrate docker container, docker compose volumes, docker container, docker migration, docker persistent storage, docker storage, docker tutorial, docker tutorial for beginners, docker volume, migrate home assistant to docker, how to move docker image to another machine, learn docker, move docker to another host, what is docker volume

Id: ZEy8iFbgbPA

Length: 8min 55sec (535 seconds)

Published: Tue Sep 14 2021