Logging with Docker
In a typical Docker environment you’ll have plenty of containers (probably in multiple networks?) on the same machine. Let’s assume, you need to debug some problems of a container, eg. because it doesn’t send mails anymore.. What would you do? Correct, you’d go and check the logs.
By default, Docker logs the messages of every container into a json file.
On a Debian-based system you’ll probably find the file at
However, to properly look into the logs you would use Docker’s logs tool.
This will print the logs, just as you would expect
cat to dump the logs in
docker-logs can also filter for time spans using
--until, and it is able to emulate a
tail -f with
However, the logs are only available for exsiting containers.
That means, if you recreate the application (i.e. you recreate the container), you’ll typically loose the log history…
If your workflow includes the
--rm, you will immediately trash the log of a container when it’s stopped.
Fortunatelly, Docker provides other logging drivers, to e.g. log to AWS, fluentd, GPC, and to good old syslog! :)
Here I’ll show how to use the host’s syslog to manage the logs of your containers.
Log to Syslog
Telling Docker to log to the host’s syslog is really easy.
You just need to use the built-in
Voilà, the container will log to the syslog and you’ll probably find the messages in
Here is an example of an Nginx, that I just started to serve my blog on my laptop:
By default, the syslog driver uses the container’s ID as the syslog tag (here it is
but you can further configure the logging driver and, for example, set a proper syslog tag:
This way, it is easier to distinguish between messages from different containers and to track the logs of an application even if the container gets recreated:
Here, I configured an nxinx that just serves the contents from
The interesting part is, however, that the container uses the
syslog driver and the syslog tag
I always prefix the tag with
docker/, to distinguish between log entries of the host machine and entries from Docker containers..
Store Docker logs seperately
The workaround so far will probably substantially spam your
/var/log/syslog, which may become very annoying… ;-)
Therefore, I recommend to write Docker’s logs to a seperate file. If you’re for example using Rsyslog, you may want to add the following configuration:
Just dump the snippet to a new file
/etc/rsyslog.d/docker.conf and restart Rsyslog.
This rule tells Rsyslog to write messages that are tagged with
/var/log/docker, and not to the default syslog file anymore.
/var/log/syslog stays clean and it’s easier do monitor the Docker containers.
Disentangle the Container logs
Since version 8.25, Rsyslog can also be used to split the docker logs into individual files based on the tag.
So you can create separate log files, one per container, which is even cleaner!
The idea is to use the tag name of containers to implement the desired directory structure.
That means, I would tag the webserver of a website with
docker/website/webserver and the database with
We can then tell Rsyslog to allow slashes in program names (see the programname section at www.rsyslog.com/doc/master/configuration/properties.html) and create a template target path for Docker log messages, which is based on the programname:
Using that configuration, our website will log to
Neat, isn’t it? :)
Even though all the individual logfiles will be smaller than a combined one, they will still grow in size. So we should tell logrotate of their existence!
Fortunatelly, this is easy as well.
Just create a new file
/etc/logrotate.d/docker containing something like the following:
This will rotate the files ending in
/var/log/docker/ and its subdirectories everyday and keep compressed logs for 7 days. Here I’m using a maximum depth of 3 subdirectories – if you need to create a deeper hierarchy of directories just add another
/var/log/docker/*/*/*/*.log etc to the beginning of the file.
- config (21) ,
- job (10) ,
- log (6) ,
- monitoring (4) ,
- network (78) ,
- google (13) ,
- amazon (1) ,
- nginx (3) ,
- docker (16) ,
- rsyslog (1) ,
- logrotate (1)