Configuring Docker Syslog Logging Driver for Docker Daemon & Containers

This article was originally written by me for SigNoz.

Logs are useful for troubleshooting and identifying issues in applications, as they provide a record of events and activities. However, managing log data can be challenging due to the large volume of log events generated by modern applications, as well as the need to balance the level of detail in the logs and the impact on the application's performance.

Collecting logs from Docker can be challenging when running a large number of containers or running Docker on multiple hosts. These challenges include managing a large volume of logs, accessing logs from multiple hosts, ensuring the security of logs, and getting a comprehensive view of container and application behavior. A centralized logging system can help address these challenges by allowing you to store and manage all of your logs in a single location.

Docker Syslog is a built-in logging system provided by Docker that allows you to centralize and manage the logs produced by your Docker containers. In this article, we will delve into the capabilities of Docker Syslog, discuss how to configure and use it as a centralized logging solution for your Docker containers and demonstrate how it can be utilized to effectively manage and analyze your Docker logs.

Understanding Syslog

Syslog stands for System Logging Protocol. Syslog is a logging protocol that enables the collection and management of log messages from various devices and systems in a central location for monitoring and resolving issues on servers and networks. It is also the standard protocol used to send system log or event messages to a specific server called a Syslog server.

A syslog server is a network server that collects and stores log messages from devices and applications in a centralized location. It uses the syslog protocol, which is a standard for sending log messages over a network, to receive log messages from various sources and store them in a database or log file for further analysis and management.

Syslog servers are commonly used to monitor and troubleshoot issues, as well as to comply with regulatory and security requirements. There are many Syslog servers available, including open-source options such as Rsyslog, Syslog-ng, and commercial solutions like Syslog-server.

What is Docker Syslog?

Docker Syslog is a logging driver for Docker that allows you to send container logs to a syslog server. It is an integral part of the Docker engine, providing a native and reliable way to manage the logs from your Docker containers.

With Docker Syslog, you can centralize and manage the logs from your Docker containers in a single place, making it easier to monitor and troubleshoot issues. This means that when Docker runs applications, the log messages produced by those applications can be sent to the Syslog server for centralized storage and analysis.

Docker Syslog integrates with Syslog servers, allowing you to use the tools provided by them to manage and analyze the logs. It also offers customization options, such as the ability to specify the log format and Syslog facility, allowing you to tailor the logging behavior to fit your specific needs.

Setting up Docker Syslog

The syslog logging driver can be setup for both the Docker daemon and for containers.

Setting up Syslog Logging driver for Docker Daemon

To configure the Docker Daemon to the Syslog driver:

Step 1: Go to the Docker daemon configuration file location:
On Linux: /etc/docker/daemon.json directory

On Windows: C:\ProgramData\docker\config\daemon.json

To use the Syslog driver as the default logging driver for Docker on Linux, you need to set the log-driver and log-opt.

Step 2: Setup the logging driver

{  "log-driver": "syslog"}

Step 3: Add options for the Syslog logging driver by including the log-opts key.
These options may include Syslog-address, Syslog-facility, and Syslog-format. This will set the Syslog driver as the default logging driver for the Docker daemon. The Docker daemon's logs will be written to the syslog server according to the specified configuration.

This will set the Syslog driver as the default logging driver for the Docker daemon. The Docker daemon's logs will be written to the Syslog server according to the specified configuration.

An example is shown below,

{  "log-driver": "syslog",  "log-opts": {    "syslog-address": "tcp://127.0.0.1:514",  }}

The log-driver and log-opt options are used to configure the logging driver for Docker. The logging driver determines how Docker handles log messages from containers and how it stores or forwards them.

• The log-driver option specifies the logging driver to use
• The log-opts option allows you to pass additional options to the logging driver. In the above configuration, the additional option passed in is the syslog-address option.
• The syslog-address option is used to specify the address of a syslog server. Docker will send the container's log output to the specified Syslog server.

There are more log-opts options which can be applied to fit your specific needs

{  "log-driver": "syslog",  "log-opts": {    "syslog-address": "tcp://127.0.0.1:514",    "syslog-facility": "daemon",    "syslog-format": "rfc5424micro"  }}

To view a complete list of the log-opts options available, visit the official docker documentation page.

Step 4: Configure and save the daemon.json file, then restart the Docker daemon to apply the changes.

sudo systemctl daemon-reloadsudo systemctl restart docker

Setting up Syslog Logging driver for Docker Containers

If you decide to configure the syslog driver for individual or specific containers, you can use the log-driver and log-opt options when starting the container.

Examples as shown below

docker run \\      --log-driver syslog \\      --log-opt syslog-address=udp://1.2.3.4:1111 \\      alpine echo hello world

docker run \\       --log-driver syslog \\       --log-opt syslog-address=tcp://syslog_server_host:514 \\       --log-opt syslog-facility=daemon \\       --log-opt syslog-format=json image_name

• The syslog-facility option is used to specify the syslog facility value that should be associated with the container's log messages. The Syslog facility value is a numeric or symbolic value that identifies the source of the log message and is used to categorize and filter the log data. Sample values that can be used include auth, cron, daemon, kern, etc.

• The syslog-format option is used to specify the format in which the container's log messages should be sent to the Syslog server. Example of more formats that could be used in Docker include json, rfc3164, rfc5424, text, etc.

Final Thoughts

In this article, we discussed Syslog, Docker Syslog as a logging driver, and how to set it up for Docker daemon and containers.

It is important to have a separate log management platform that provides additional capabilities and flexibility for managing and analyzing the Syslog logs from your Docker containers.

A centralized log management tool can also help to ensure that you have a robust and scalable solution for log analytics that meets your specific needs and requirements, as the Syslog server or logging driver may not have the necessary features or capabilities to fully manage and analyze the logs. For example, you may want to perform complex log parsing, filtering, or transformation operations that are not possible with the Syslog server or logging driver.

An advanced centralized logging platform/tool for collecting your logs is SigNoz - an open-source log management solution.

Docker Logs analysis with SigNoz

SigNoz is a full-stack open-source Application Performance Monitoring tool that you can use for monitoring logs, metrics, and traces. One key aspect of observability is log management, and SigNoz provides a range of tools for collecting, analyzing, and visualizing Docker logs.

It uses ClickHouse, a columnar database, to efficiently store and provide access to log data for analysis.

SigNoz uses OpenTelemetry for instrumenting applications. OpenTelemetry, backed by CNCF, is quickly becoming the world standard for instrumenting cloud-native applications.

The logs tab in SigNoz has advanced features like a log query builder, search across multiple fields, structured table view, JSON view, etc.

SigNoz offers real-time analysis of logs, enabling you to search, filter, and visualize them as they are generated. This can assist in identifying patterns, trends, and problems in the logs and resolving issues efficiently.

With advanced Log Query Builder, you can filter out logs quickly with a mix and match of fields.

Getting started with SigNoz

SigNoz can be installed on macOS or Linux computers in just three steps by using a simple install script.

The install script automatically installs Docker Engine on Linux. However, on macOS, you must manually install Docker Engine before running the install script.

git clone -b main https://github.com/SigNoz/signoz.gitcd signoz/deploy/./install.sh

You can visit our documentation for instructions on how to install SigNoz using Docker Swarm and Helm Charts.

Related Posts
Docker Logging Complete Guide - Configuration and Logging Strategies
Docker Log Rotation Configuration Guide