Rsyslog

Rsyslog is a rocket-fast system for log processing.

It offers high-performance, great security features, and a modular design. While it started as a regular syslogd, rsyslog has evolved into a kind of swiss army knife of logging, being able to accept inputs from a wide variety of sources, transform them, and output to the results to diverse destinations.

It can deliver over one million messages per second to local destinations when limited processing is applied (based on v7, December 2013). Even with remote destinations and more elaborate processing the performance is usually considered “stunning”.

It has a strong enterprise focus but also scales down to small systems. It supports, among others, MySQL, PostgreSQL, failover log destinations, ElasticSearch, syslog/tcp transport, fine grain output format control, high precision timestamps, queued operations and the ability to filter on any message part.

Feature

• Multi-threading
• TCP, SSL, TLS, RELP
• MySQL, PostgreSQL, Oracle and more
• Filter any part of syslog message
• Fully configurable output format
• Suitable for enterprise-class relay chains

Changelog v8.2310

– 2023-10-04: Add CAP_NET_RAW capability due to the omudpspoof module
The CAP_NET_RAW ensures the use of RAW and PACKET sockets,
which is utilized by the omudpspoof module, more precisely
the libnet_init function.
Thanks to Attila Lakatos for the patch.
– 2023-10-04: Add new global config option “libcapng.enable”
Defines whether rsyslog should drop capabilities at startup or not.
By default, it is set to “on”. Until this point, if the project was
compiled with –enable-libcap-ng option, capabilities were
automatically dropped. This is configurable now.
Thanks to Attila Lakatos for the patch.
– 2023-10-04: tcp net subsystem: handle data race gracefully
It may happen that a socket file descriptor has been closed either
while setting up poll() et al or while being inside the system call.
This was previously treated as error and caused abort in debug
builds. However, it was essentially ignored in production builds.
This has now been fixed and now is always gracefully ignored. This
most importantly fixes some flakes in CI runs (which were caused
by this situation).
– 2023-09-29: imrelp bufgifx: avoid crash on restart in imrelp SIGTTIN handler
While existing, if at specific time rsyslog receives a SIGTTIN, it
crashes due to 2 issues.
1. debug.unloadModules=”off” a double free of pRelpEngine
2. debug.unloadModules=”on” it crashes because the signal handler has
been unmapped from memory.
This patch covers both issues.
Thanks to Ali Abdallah for the patch.
– 2023-09-28: fix startup issue on modern systemd systems
When we startup AND are told to auto-background ourselfs, we must
close all unneeded file descriptors. Not doing this has some
security implications. Traditionally, we do this by iterating
over all possible file descriptor values. This is fairly compatible,
because we need no OS-specific method. However, modern systemd configs
tend to not limit the number of fds, so there are potentially 2^30(*)
fds to close. While this is OKish, it takes some time and makes
systemd think that rsyslog did not properly start up.
We have now solved this by using the /proc filesystem to obtain our
currently open fds. This works for Linux, as well as Cygwin, NetBSD,
FreeBDS and MacOS. Where not available,and close_range() is available
on the (build) platform, we try to use it. If that fails as well, we
fall back to the traditional method. In our opionion, this fallback
is unproblematic, as on these platforms there is no systemd and in
almost all cases a decent number of fds to close.
Very special thanks go out to Brennan Kinney, who clearly described
the issue to us on github and also provided ample ways to solve it.
What we did is just implement what we think is the best fit from
rsyslog’s PoV.
(*) Some details below on the number of potentially to close fds.
This is directly from a github posting from Brennan Kinney.
Just to clarify, by default since systemd v240 (2018Q4), that
should be `1024:524288` limit. As in the soft limit is the expected
`1024`.
The problem is other software shipping misconfiguration in systemd
services that overrides this to something silly like
`LimitNOFILE=infinity`.
– Which will map to the sysctl `fs.nr_open` (_a value systemd
v240 also raises from `2^20` to 2^30`, some distro like Debian are
known to opt-out via patch for the `fs.nr_open` change_).
– With the biggest issue there being that the soft limit was also
set to `infinity` instead of their software requesting to raise
the soft limit to a higher value that the hard limit permits.
`infinity` isn’t at all sane though.
– The known source of this misconfiguration is container software such
as Docker and `containerd` (_which would often sync with the
systemd `.service` config from the Docker daemon `dockerd.service`_).
closes https://github.com/rsyslog/rsyslog/issues/5158
– 2023-09-13: Add the ‘batchsize’ parameter to imhiredis
Parameter set to allow configuring the amount of entries imhiredis debatches at once.
Default value of ’10’ has been kept to avoid any side effect on existing
configurations.
Thanks to Jérémie Jourdin for the patch.
– 2023-09-13: omprog bugfix: Add CAP_DAC_OVERRIDE to the bounding set
The omprog module uses the execve() function to execute
a third party program. Some required capabilities were not
preserved in the bounding set [1]. This caused problems, e.g.
the program could not write to files even if rsyslog was
executed as root and privileges were not dropped. As of now,
only the CAP_DAC_OVERRIDE capability is added to the bounding
set. Others could be added later, if there is justification
behind that.
[1] The capability bounding set is a security mechanism that
can be used to limit the capabilities that can be gained
during an execve(2). During an execve, the capability
bounding set is ANDed with the file permitted capability
set, and the result of this operation is assigned to the
thread’s permitted capability set. The capability
bounding set thus places a limit on the permitted
capabilities that may be granted by an executable file.
Thanks to Attila Lakatos for the patch.
– 2023-09-13: tcpflood bugfix: plain tcp send error not properly reported
The error code when plain tcp sending failed was improperly returned,
resulting in no meaningful error message.
Note: tcpflood is a testbench tool, not part of production rsyslog.

Download & Use

Copyright (C) Rainer Gerhards <rgerhards@adiscon.com> lead rsyslog developer