Most people expect to see how many log messages are waiting in the disk-buffer from the size of the syslog-ng disk-buffer file.. While it was mostly true for earlier syslog-ng releases, for recent syslog-ng releases (3.34+) the disk-buffer file can stay large even when it is empty. This is a side effect of a recent syslog-ng performance tuning.
Before you begin
As mentioned earlier in the disk-buffer blog series, you need an up-to-date syslog-ng version for the latest performance improvements. In this case, the feature was introduced in version 3.33, and received some fine-tuning in version 3.34. If your OS has an earlier syslog-ng version available, check the 3rd party download page if there are up-to-date syslog-ng packages available for your OS: https://www.syslog-ng.com/products/open-source-log-management/3rd-party-binaries.aspx
Why?
You always have to make choices: cheap or good, and so on. This also seems to be the case when using the disk-buffer.. Earlier versions of the disk-buffer used to have a low performance. One of the reasons for this was that syslog-ng tried to free up the disk space as soon as the log message was sent successfully. It is good from the disk space point of view, however, it also requires a lot higher CPU usage and I/O activities, thus slowing the whole system down. Recent syslog-ng versions are lazier, when it comes to freeing disk space.
There are two conflicting aspects here. In an ideal world, you do not need the disk-buffer at all, so a large disk-buffer file is just a waste of disk space. However, when it comes to a networking problem and the destination is unavailable, the disk-buffer comes in handy. Plus a large, continuous, pre-allocated file works a lot faster than a file with a continuously changing size.
The current solution is that syslog-ng frees up disk-space once messages are sent out, but it is done slowly, in larger chunks. The default setting is that syslog-ng truncates the disk-buffer file when it can free up at least 10% of the file at once. For more information on how to change the truncation ratio, see the documentation. Setting a smaller value results in a quicker clearing of disk space, whereas a larger ratio provides better performance.
What is next?
Version 4.0 of syslog-ng is right around the corner. There are discussions among syslog-ng developers, that the default setting should be changed, so that the disk-buffer file is never truncated by default. Of course, in an ideal case it is just a huge waste of disk space. However, if the disk-buffer is needed, this setting gives the best performance.
What do you think about this change? You should write your opinion at https://github.com/syslog-ng/syslog-ng/discussions/3883
-
If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @Pczanik.