Reading files and monitoring directories became a lot more efficient in recent syslog-ng releases. However, it is also needed manual configuration. Version 4.11 of syslog-ng can automatically configure the optimal setting for both.
If you follow my blogs, you might have already seen that syslog-ng file reading and directory monitoring received huge performance gains while reducing resource usage. You can read about these in the following blog posts:
-
FreeBSD / MacOS: https://www.syslog-ng.com/community/b/blog/posts/version-4-8-0-of-syslog-ng-improves-freebsd-and-macos-support
You can find a few more options in the documentation, however, those only work if you use the modified Ivykis library bundled with the syslog-ng sources. Packages in FreeBSD and various Linux distributions tend to use unmodified upstream Ivykis. See https://syslog-ng.github.io/admin-guide/060_Sources/021_Wildcard-file/001_File_following for details.
Before you begin
Automatic configuration of the wildcard-file() source arrived to syslog-ng right before the 4.11 release. To test it, you need version 4.11 of syslog-ng, or a recent enough development snapshot or nightly build.
Configuring syslog-ng
To preserve backward compatibility, automatic configuration of file reading and directory monitoring is not enabled by default. You have to configure it explicitly using the following two options in the wildcard-file() source:
monitor-method(“auto”) follow-method(“auto”)
These settings will configure the wildcard-file() source to use the optimal methods based on the platform and available technologies.
The default follow-method() and poll method are both legacy, both of which also used the follow-freq() option. If you switch to use the auto method and follow-freq() is set to a non-zero value, you will see a warning message that the value is ignored when you start syslog-ng.
What is next?
We recommend using the new “auto” settings for any new configurations. It also allows you to use the same configuration on multiple platforms, as you do not have to specify platform-specific options anymore. So, rewriting old configurations might also be useful.
-
If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @PCzanik, on Mastodon as @Pczanik@fosstodon.org.