Actions

Tags

Peter Czanik
16 Aug 2022
Type support: working with sudo logs in syslog-ng 4.0

Last week I gave you a quick introduction to a major syslog-ng 4.0 feature: type support. I mentioned that it also works nicely for JSON-formatted sudo logs. I have been asked to share a working syslog-ng configuration.

From this blog, you can learn how to turn on JSON-formatted logging in sudo, and how to work with those logs in syslog-ng.

Before you begin

You need pretty recent sudo and syslog-ng versions to test it. On the sudo side, you need at least version 1.9.4, which is available in BSD ports, in rolling Linux distributions and in some recent releases. You can also find builds for some popular platforms on the sudo website at: https://www.sudo.ws/getting/download/#binary

On the syslog-ng side, you need at least version 3.37, but a recent git snapshot build is recommended:

Configuring sudo

To enable JSON-formatted logging, open the sudoers file using visudo and add the following line:

Defaults log_format=json

Configuring syslog-ng

Append this to syslog-ng.conf, or create a new .conf file under /etc/syslog-ng/conf.d/ if syslog-ng on your platform is configured to use this directory:

# sudo JSON logs
filter f_sudo {
  program(sudo);
};
destination d_sudo {
  file("/var/log/sudo" template("$(format-json --scope rfc5424 --scope dot-nv-pairs
       --rekey .* --shift 1 --scope nv-pairs --exclude MESSAGE --exclude .journal*)\n\n"));
};
log {
  source(src);
  filter(f_sudo);
  destination(d_sudo);
};

The filter selects sudo log messages. This way, it is easier to find relevant log messages in the log file.

The file destination uses JSON formatting, and it has name-value pairs for syslog header fields and includes data parsed from sudo log messages. So, this file does not store the original incoming JSON message, but it is created from name-value-pairs parsed from messages.

The log path uses a source for local logs (defined in syslog-ng.conf), selects log messages from sudo and stores results in the file destination. What you do not see in the log path is a parser to parse sudo log messages. The system() source of syslog-ng automagically parses log messages from sudo, both traditional and JSON formatted.

Note that the source name in the log path is specific to your syslog-ng.conf. Find the source that uses the system() source for local log messages.

Enabling syslog-ng 4.0 mode

The syslog-ng configuration starts with a version number declaration. For syslog-ng version 3.37, it looks like:

@version: 3.37

If the syslog-ng behavior has changed between versions, the version string defines the behavior.
This is how you can enable the 4.0 functions:

@version: 4.0

The 3.37 release already allows you to test some of the 4.0 features, but I recommend using one of the latest git snapshots.

Testing

Once you have both sudo and syslog-ng configured, you are ready for testing. First, test the 3.X mode, then 4.0. You can generate a log message with sudo, just try to run anything through sudo:

sudo ls

In /var/log/messages, where log messages are stored unprocessed, you will see a log message like this:

Aug 11 12:48:00 tumbleweed sudo[2782]: @cee:{"sudo":{"accept":{"uuid":"fe1d4b3adc-a9af-4193-1708-35aaa1f2f4","server_time":{"seconds":1660214880,"nanoseconds":281301380,"iso8601":"20220811104800Z","localtime":"Aug 11 12:48:00"},"submit_time":{"seconds":1660214880,"nanoseconds":276503916,"iso8601":"20220811104800Z","localtime":"Aug 11 12:48:00"},"iolog_path":"/var/log/sudo-io/00/00/1U","submituser":"root","command":"/usr/bin/ls","runuser":"root","runcwd":"/root","ttyname":"/dev/pts/0","submithost":"tumbleweed","submitcwd":"/root","runuid":0,"columns":80,"lines":24,"runargv":["ls"],"runenv":["LANG=POSIX","TERM=xterm-256color","LC_CTYPE=en_US.UTF-8","MAIL=/var/mail/root","PATH=/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin:/usr/local/sbin","LOGNAME=root","USER=root","HOME=/root","SHELL=/bin/bash","SUDO_COMMAND=/usr/bin/ls","SUDO_USER=root","SUDO_UID=0","SUDO_GID=0"]}}}

As you can see, there are no quotes around numbers, which, in JSON formatting means that it is not a text but a number (actually it could be any of a few more data types, like boolean).

If still in version 3.X mode, you can see something similar in the parsed and reconstructed JSON logs:

{"cee":{"sudo":{"accept":{"uuid":"fe1d4b3adc-a9af-4193-1708-35aaa1f2f4","ttyname":"/dev/pts/0","submituser":"root","submithost":"tumbleweed","submitcwd":"/root","submit_time":{"seconds":"1660214880","nanoseconds":"276503916","localtime":"Aug 11 12:48:00","iso8601":"20220811104800Z"},"server_time":{"seconds":"1660214880","nanoseconds":"281301380","localtime":"Aug 11 12:48:00","iso8601":"20220811104800Z"},"runuser":"root","runuid":"0","runenv":"LANG=POSIX,TERM=xterm-256color,LC_CTYPE=en_US.UTF-8,MAIL=/var/mail/root,PATH=/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin:/usr/local/sbin,LOGNAME=root,USER=root,HOME=/root,SHELL=/bin/bash,SUDO_COMMAND=/usr/bin/ls,SUDO_USER=root,SUDO_UID=0,SUDO_GID=0","runcwd":"/root","runargv":"ls","lines":"24","iolog_path":"/var/log/sudo-io/00/00/1U","command":"/usr/bin/ls","columns":"80"}}},"app":{"name":"cee"},"SOURCE":"src","PROGRAM":"sudo","PRIORITY":"notice","PID":"2782","HOST_FROM":"tumbleweed","HOST":"tumbleweed","FACILITY":"auth","DATE":"Aug 11 12:48:00"}

If you take a closer look, you will see that all values are enclosed in quotes. It means that no matter what the type of the name-value pair in the incoming log message was, it is treated as text by syslog-ng.

Now, rewrite the version string to 4.0 in syslog-ng.conf:

@version: 4.0

Once you reloaded syslog-ng with the new configuration, run sudo again. This time, the log message will look slightly different. We still interpret numbers as numbers, independent of the quotes. However, for applications, proper typing is important. In version 4.0, syslog-ng keeps the type information from JSON formatted log messages:

{"cee":{"sudo":{"accept":{"uuid":"37633297e5-23b2-4981-84f5-54494818d5","ttyname":"/dev/pts/0","submituser":"root","submithost":"tumbleweed","submitcwd":"/root","submit_time":{"seconds":1660214895,"nanoseconds":729168406,"localtime":"Aug 11 12:48:15","iso8601":"20220811104815Z"},"server_time":{"seconds":1660214895,"nanoseconds":731882223,"localtime":"Aug 11 12:48:15","iso8601":"20220811104815Z"},"runuser":"root","runuid":0,"runenv":["LANG=POSIX","TERM=xterm-256color","LC_CTYPE=en_US.UTF-8","MAIL=/var/mail/root","PATH=/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin:/usr/local/sbin","LOGNAME=root","USER=root","HOME=/root","SHELL=/bin/bash","SUDO_COMMAND=/usr/bin/ls","SUDO_USER=root","SUDO_UID=0","SUDO_GID=0"],"runcwd":"/root","runargv":["ls"],"lines":24,"iolog_path":"/var/log/sudo-io/00/00/1V","command":"/usr/bin/ls","columns":80}}},"app":{"name":"cee"},"SOURCE":"src","PROGRAM":"sudo","PRIORITY":"notice","PID":"3038","HOST_FROM":"tumbleweed","HOST":"tumbleweed","FACILITY":"auth","DATE":"Aug 11 12:48:15"}

The difference is not huge, the values of lines and columns are not enclosed in quotation marks, but it means that in a log analysis software you see these values as numbers, not as text.

What is next?

Now that you have seen that syslog-ng can handle data types from parsed messages, it is time to store log messages to somewhere, where typing actually matters. I stored logs to an OpenSearch database using the elasticsearch-http() destination of syslog-ng. I changed the d_sudo() destination:

destination d_sudo {
      elasticsearch-http(
        index("syslog-ng")
        type("")
        url("https://localhost:9200/_bulk")
        template("$(format-json --scope rfc5424 --scope dot-nv-pairs
        --rekey .* --shift 1 --scope nv-pairs
        --exclude DATE @timestamp=${ISODATE})")
        user("admin")
        password("***")
        ca-file("/etc/syslog-ng/tls/root-ca.pem")
    );
};

-

If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @PCzanik.

Related Content