The difference between throttle() and rate-limit() in syslog-ng

There are multiple ways in syslog-ng to limit message rate. The throttle() option of syslog-ng destinations tries to make sure that all messages are delivered without exceeding a specified message rate. The rate-limit() filter introduced in syslog-ng 3.36 drops surplus log messages, making sure that a processing pipeline or destination is not overloaded with log messages.

Before you begin

If you just need the throttle() option in a syslog-ng destination, then you most likely do not need to install the latest syslog-ng version. Even RHEL 7 has version 3 of syslog-ng available in EPEL. However, to use the rate-limit() filter, you need to use at least syslog-ng 3.36. While it is not yet available in most Linux distributions, there is a good chance that you can find it in a third-party repository:

Using throttle()

The throttle() option of various syslog-ng destinations makes sure that the message rate is throttled down to a value acceptable by the specific destination. As in-memory buffers can fill up quickly, you are advised to combine the throttle() option with a disk-buffer. This way, no messages are lost if there is a sudden spike in message rate. Some destinations are throttled by default, like discord(), where a rate of more than a couple messages per second is considered flooding.

Adding throttling to a destination is easy, just do not forget to also add a disk-buffer. Otherwise, there is a good chance for message loss. Here is a simple example allowing 100 messages a second over a TCP connection, combined with a larger disk-buffer.

destination d_throttled {

Using rate-limit()

The rate-limit() filter was originally called throttle(), but it was renamed to rate-limit() to avoid confusion. This is because there is a fundamental difference between the throttle() option of destinations and the rate-limit() filter: here, the message loss is intended. The rate-limit() filter can be used as a precaution: ensuring that a specific resource cannot be flooded with messages. You can set a simple limit: in this case, any message above the specified rate is simply discarded. There is also a possibility to configure a template using macros. For example, using the $HOST macro, the message rate is calculated for each host separately. This way, a single host cannot overload a message pipeline with a peak of messages, as messages above the rate-limit() are discarded.

filter f_rate_limit {

If you put this filter in the log path on a relay forwarding log messages from five hosts to a central syslog-ng server, it makes sure that no hosts can send more than 5000 messages a second. Surplus log messages are discarded. In this case, this means that the message rate does not exceed 25000 messages a second.


Testing the template() part of the rate-limit() filter is not easy, but testing the rate-limit() filter can easily be done using a network source and loggen. Append this configuration to syslog-ng.conf or place it with a .conf extension in the /etc/syslog-ng/conf.d, if using it is supported on your system.

source s_tcp {
filter f_rlimit {
destination d_file {
log {

Once you reloaded syslog-ng, you should have a new tcp() source listening, and logs written to a file. Only 100 log messages are written to the file a second, the rest are discarded. Loggen sends around a thousand messages a second by default:

czplaptop:/etc/syslog-ng/conf.d # loggen -i -S localhost 514
count=1803, rate = 907.78 msg/sec
count=2255, rate = 903.76 msg/sec
count=2711, rate = 911.80 msg/sec
count=9959, rate = 899.70 msg/sec
average rate = 995.62 msg/sec, count=9959, time=10.0028, (average) msg size=256, bandwidth=248.90 kB/sec
czplaptop:/etc/syslog-ng/conf.d # wc -l /var/log/myfile
1005 /var/log/myfile

As you can see, 9959 messages were sent in 10.0028 seconds, but only 1005 were written to disk, which means that the limit on message rate was kept perfectly.


If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at On Twitter, I am available as @PCzanik.

Related Content