Even with Docker you need to care about backups.. ;-)
As you usually mount all the persistent data into the container the files will actually be on your host. Thus, you can simply do the backup of these files. However, for MySQL/MariaDB I prefer having an actual SQL-dump. Therefore I just developed the Docker MySQL-Backup tool. You will find the sources at the corresponding GitHub repository.
How does Docker MySQL-Backup work?
The tool basically consists of two scripts:
- a config file in
/etc/default/docker-mysql-backupto setup the path for the backup location and the path to gzip,
- the script
/etc/cron.daily/docker-mysql-backupwhich does the actual job.
/etc/cron.daily/docker-mysql-backup parses the output of the
docker ps command to find running containers of the MySQL image.
More precisely, it looks for containers of images that start with either
The actual filter command is
That of course only matches the original MySQL/MariaDB image names (if you have a good reason to derive an own version of that image please tell me!).
For every matching
$container the script will exec the following command:
With the following variables:
$BACKUP_DIRis a concatenation of
/etc/default/docker-mysql-backup) and the container name,
$NOWis the current time stamp as
Thus, the backups are compressed, organised in subdirectories of
$BACKUP_BASE, and the SQL-dumps have a time stamp in their names.
$BACKUP_BASE defaults to
/srv/backup/mysql/, but can be configured in
Last but not least, the script also cleans the backups itself.
It will keep the backups of the last 30 days and all backups of days that end with a
So you will keep the backups from the 2nd, the 12th, and the 22nd of every month.
As the script is stored in
/etc/cron.daily/ the cron tool will execute the backup script on a daily basis.
Restore a dump
Restoring the dump is quite easy.
Let’s assume your container’s name is
$container and the dump to restore carries the time stamp
Then you just need to run:
This will mount the backup directory in
/srv of the running container and then decompress and import the SQL-dump on the fly.
Manual installation through GitHub
Clone the Docker MySQL-Backup repository:
Copy the backup script to the
cron.daily (most likely
/etc/cron.daily/) directory on your system:
Copy the configuration to
Installation from my Apt repository
If you’re running a Debian-based system you may want to use my apt-repository to install the Docker MySQL-Backup tool. In that case you just need to run
Afterwards, look into
/etc/default/docker-mysql-backup for configuration options.
This way, you’ll always stay up-to-date with bug fixes and new features :)
Docker is cool. Jails tools into containers. That of course sounds clean and safe and beautiful etc. However, the tools are still buggy and subject to usual attacks, just as they were running on your main host! Thus, you still need to make sure your containers are up to date.
But how would you do that?
Approaches so far
On the one hand, let’s assume you’re using Docker Compose, then you can go to the directory containing the
docker-compose.yml and call
However, this will just update the images used in that Docker Compose setup – all the other images on your system wouldn’t be updated. And you need to do that for all Docker Compose environments. And if you’re running 30 containers of the same image it would check 30 times for an update of that image – quite a waste or power and time..
On the other hand, you may use the dupdate tool, introduced earlier:
It is able to go through all your images and update them, one after the other.
That way, all the images on your system will be updated.
dupdate doesn’t know about running containers.
Thus, currently running tools and services won’t be restarted..
Better: Docker Auto-Update
Therefore, I just developed a tool called Docker Auto-Update that combines the benefits of both approaches.
It first calls
dupdate -s to update all your images and then iterates over a pre-defined list of Docker Compose environments to call a
docker-compose up -d --remove-orphans.
The tool consists of three files:
/etc/cron.daily/docker-updaterreads the configuration in
/etc/default/docker-updaterand does the regular update
/etc/default/docker-updaterstores the configuration. You need to set the
1, otherwise the update tool won’t run.
/etc/docker-compose-auto-update.confcarries a list of Docker Compose environments. Add the paths to the
docker-compose.ymlfiles on your system, one per line
As it’s installed in
/etc/cron.daily/, cron will take care of the job and update your images and containers on a daily basis.
If your system is configured properly, cron will send an email to the systems administrator when it updates an image or restarts a container.
You see, no magic, but a very convenient workflow! :)
To install the Docker Auto-Update tool, you may clone the git repository at GitHub. Then,
- move the
- move the
./etc/default/docker-updaterconfig file to
- update the setup in
/etc/default/docker-updater– at least set
- create a list of Docker Compose config files in
/etc/docker-compose-auto-update.conf- one path to a
If you’re using a Debian based system you may install the Docker-Tools through my apt-repository:
/etc/default/docker-updater and at least set
This way, you’ll stay up-to-date with bug fixes etc.
The tool will update your images and containers automatically – very convenient but also dangerous! The new version of an image may break your tool or may require an updated configuration.
Let’s assume you rendered your FreeBSD system unbootable.. Yeah, happens to the best, but how can you still copy the data stored on a ZFS to another machine? You probably just shouted RSYNC - but it’s not that easy.
You would need a FreeBSD live os (either on a USB pen drive or on a CD/DVD) and boot into that system. However, by default you do not have network, the ZPool is not mounted, there is no rsync and SSH is not running, and the live os is not writable, which brings another few issues…
This is a step-by-step how-to through all the obstacles. Just boot into your live os (get it from freebsd.org) and go on with the following…
By default your live system does not have networking setup correctly.
ifconfig to see if the network interface is up. If it’s not you can bring it up using:
(assuming your inteface is called em0)
If it is up, you need to configure it. When you’re using a DHCP server you can just ask for an IP address using:
Otherwise you need to configure the addresses manually:
Afterwards you should be able to ping other machines, such as
Mount the ZPool
Your ZPool won’t be mounted by default; you need to do it manually. To list all pools available on that machine just call:
This searches through the devices in
/dev to discover ZPools. You may specify a different directory with
-d (see man page for zpool).
To actually import and mount your ZPool you need to provide its name, for example:
This will import the ZPool
zroot. Moreover, the argument
-o altroot=/mnt will mount it to
/mnt instead of
/ and the
-f will mount it even if it may be in use by another system (here we’re sure it isn’t, aren’t we?).
Create some Writeable Directories
The next problem is, that you do not have permissions to write to
/etc, which you need to e.g. create SSH host keys etc.
However, that’s also not a big issue as we have the
unionfs filesystem! :)
UnionFS will mount a directory as an overlay over another directory.
Let’s assume you have some space in
$SPACE (maybe in the ZPool that you just mounted or on another USB drive), then you can just create a few directories:
and mount it as
unionfs to the root’s equivalents:
Now we can write to
/etc, while the actual changes will be written to
$SPACE/etc! Isn’t that a great invention?
Start the SSH service
/etc is writable we can start caring about the SSH daemon.
First, we need to configure it to allow root to login.
Add the follwing line to the
Then, we can start the ssh daemon using:
It will automatically create host keys and all the necessary things for a first start of SSH.
If that was successful, port
22 should now be open:
Set root Password
To be able to login you of course need to set a root password:
Aftwerwards, you should be able to login through SSH from any other machine. Go ahaed and give it a try!
Install and Run rsync
Almost there, but the freeBSD live image doesn’t come with
So we need to do it manually:
This will first tell us that not even
pkg is installed, but answering the question with
y it will automatically install itself.
And as everything is mounted as UnionFS, the stuff will actually be installed to
$SPACE/... instead of
However, you should now be able to do the rsync job from where ever you want :)
Never heard of that FlexNet thing, but according to Wikipedia it’s a software license manager. And we all know how this whole DRM thing just bugs us.. So it bugged me because the new system wouldn’t boot properly.. Other people having similar problems.
However, it seems impossible to force grub overriding this sector, but you may wipe it manually. In my case sector 32 was infected by DRM, so I did the following:
If that’s done Grub installs like a charm, the system booted again, and the admin was happy that another DRM thing died :)
As I’m working with Docker quite intensively it was about time to develop some tools that help me managing different tasks. Some of them have already been existing as functions in my environment or something, but now they are assembled in a git repository at GitHub.
The toolbox currently consists of the following tools:.
dclean cleans your setup
The Docker-Clean tool
dclean helps getting rid of old, exited Docker containers.
Sometimes I forget the
--rm flag during tests, and when I realise it there are already hundreds of orhpaned containers hanging around..
dclean without arguments removes all of them quickly.
dclean tool accepts a
-i flag which will clean the images.
It will prune all dangling images.
Dangling images are orphaned and usually not needed anymore.
dclean -i will remove them.
denter gets you into a containers
The Docker-Enter tool
denter beames you into a running Docker container.
Just provide the container’s name or CID as an argument to get a
/bin/bash inside the container.
denter will just call
/bin/bash by default.
So there is no magic, it’s just a shortcut..
You may overwrite the program to be executed by providing it as a second argument.
ps -ef in the container with the id
dip shows IP addresses
The Docker-IP tool
dip shows the IP addresses of running containers.
Without arguments it will print the IP addresses, names, and container ids of all running containers.
If your interested in the IP address of a specific container you may pass that container’s CID as an argument with
-c, just like:
This will show the IP of the container with id
dkill stops all running containers
The Docker-Kill tool
dkill is able to kill all running containers.
It doesn’t care what’s in the container, it will just iterate over the
docker ps list to stop all running containers.
As this is quite dangerous, it requires a
-f flag to actually kill the containers.
You may afterwards run the
dclean tool from above to get rid of the cadavers..
dupdate updates images
The Docker-Update tool
dupdate helps you staying up-to-date.
It will iterate over all your images and tries to pull new versions of that image from the Docker registry (or your own registry, if you have one).
By default, it will echo the images that have been updates and tells you which images cannot be found (anymore) on the registry.
You may pass the
dupdate to enable verbose mode and also get a report for images that do not have a newer version at the registry.
This way, you can make sure that all images are checked.
Similarly, you can pass
-s to enable silent mode and suppress messages about images that cannot be found at the registry.
You may also want to look at the Docker-Update tool?
This way, you’ll stay up-to-date with bug fixes etc.