syslog-ng Open Source Edition 3.36 - Administration Guide

Preface Introduction to syslog-ng The concepts of syslog-ng Installing syslog-ng The syslog-ng OSE quick-start guide The syslog-ng OSE configuration file source: Read, receive, and collect log messages
How sources work default-network-drivers: Receive and parse common syslog messages internal: Collecting internal messages file: Collecting messages from text files wildcard-file: Collecting messages from multiple text files linux-audit: Collecting messages from Linux audit logs mqtt: receiving messages from an MQTT broker network: Collecting messages using the RFC3164 protocol (network() driver) nodejs: Receiving JSON messages from nodejs applications mbox: Converting local email messages to log messages osquery: Collect and parse osquery result logs pipe: Collecting messages from named pipes pacct: Collecting process accounting logs on Linux program: Receiving messages from external applications python: writing server-style Python sources python-fetcher: writing fetcher-style Python sources snmptrap: Read Net-SNMP traps sun-streams: Collecting messages on Sun Solaris syslog: Collecting messages using the IETF syslog protocol (syslog() driver) system: Collecting the system-specific log messages of a platform systemd-journal: Collecting messages from the systemd-journal system log storage systemd-syslog: Collecting systemd messages using a socket tcp, tcp6, udp, udp6: Collecting messages from remote hosts using the BSD syslog protocol— OBSOLETE unix-stream, unix-dgram: Collecting messages from UNIX domain sockets stdin: Collecting messages from the standard input stream
destination: Forward, send, and store log messages
amqp: Publishing messages using AMQP collectd: sending metrics to collectd discord: Sending alerts and notifications to Discord elasticsearch2: Sending messages directly to Elasticsearch version 2.0 or higher (DEPRECATED) elasticsearch-http: Sending messages to Elasticsearch HTTP Bulk API file: Storing messages in plain-text files graphite: Sending metrics to Graphite Sending logs to Graylog hdfs: Storing messages on the Hadoop Distributed File System (HDFS) Posting messages over HTTP http: Posting messages over HTTP without Java kafka: Publishing messages to Apache Kafka (Java implementation) kafka-c(): Publishing messages to Apache Kafka using the librdkafka client (C implementation) loggly: Using Loggly logmatic: Using Logmatic.io mongodb(): Storing messages in a MongoDB database mqtt() destination: sending messages from a local network to an MQTT broker network: Sending messages to a remote log server using the RFC3164 protocol (network() driver) osquery: Sending log messages to osquery's syslog table pipe: Sending messages to named pipes program: Sending messages to external applications pseudofile() python: writing custom Python destinations redis: Storing name-value pairs in Redis riemann: Monitoring your data with Riemann slack: Sending alerts and notifications to a Slack channel smtp: Generating SMTP messages (email) from logs snmp: Sending SNMP traps Splunk: Sending log messages to Splunk sql: Storing messages in an SQL database stomp: Publishing messages using STOMP Sumo Logic destinations: sumologic-http() and sumologic-syslog() syslog: Sending messages to a remote logserver using the IETF-syslog protocol syslog-ng(): Forward logs to another syslog-ng node tcp, tcp6, udp, udp6: Sending messages to a remote log server using the legacy BSD-syslog protocol (tcp(), udp() drivers) Telegram: Sending messages to Telegram unix-stream, unix-dgram: Sending messages to UNIX domain sockets usertty: Sending messages to a user terminal: usertty() destination Write your own custom destination in Java or Python Client-side failover
log: Filter and route log messages using log paths, flags, and filters Global options of syslog-ng OSE TLS-encrypted message transfer template and rewrite: Format, modify, and manipulate log messages parser: Parse and segment structured messages
Parsing syslog messages Parsing messages with comma-separated and similar values Parsing key=value pairs JSON parser XML parser Parsing dates and timestamps Python parser Parsing tags Apache access log parser Linux audit parser Cisco parser Parsing enterprise-wide message model (EWMM) messages iptables parser Netskope parser panos-parser(): parsing PAN-OS log messages Sudo parser Websense parser Fortigate parser Check Point Log Exporter parser Regular expression (regexp) parser db-parser: Process message content with a pattern database (patterndb)
Correlating log messages Enriching log messages with external data Statistics of syslog-ng Multithreading and scaling in syslog-ng OSE Troubleshooting syslog-ng Best practices and examples The syslog-ng manual pages Creative Commons Attribution Non-commercial No Derivatives (by-nc-nd) License Glossary

Setting severity with the set-severity() rewrite function

It is possible to configure the severity field with the set-severity() rewrite function. When configured, the set-severity() rewrite function will only rewrite the $SEVERITY field in the message to the first parameter value specified in the function.

NOTE: If the parameter value is not a valid parameter value, the function ignores it and sends a debug message, but the syslog-ng Open Source Edition (syslog-ng OSE) application still sends the message.

Declaration
rewrite <name_of_the_rule> {
    set-severity("severity string or number");
};
Parameters

The set-severity() rewrite function has a single, mandatory parameter that can be defined as follows:

set-severity( "parameter1" );
Accepted values

The set-severity() rewrite function accepts the following values:

  • numeric strings: [0-7]
  • named values: emerg, emergency, panic, alert, crit, critical, err, error, warning, warn, notice, info, informational, debug
Example usage for the set-severity() rewrite function

The following examples can be used in production for the set-severity() rewrite function.

Example using string:

rewrite {
    set-severity("info");
};

Example using numeric string:

rewrite {
    set-severity("6");
};

Example using template:

rewrite {
    set-severity("${.json.severity}");
};

Was this topic helpful?

[Select Rating]



Setting the facility field with the set-facility() rewrite function

It is possible to set the facility field with the set-facility() rewrite function. When set, the set-facility() rewrite function will only rewrite the $PRIORITY field in the message to the first parameter value specified in the function.

NOTE: If the parameter value is not a valid parameter value, the function ignores it and sends a debug message, but the application still sends the message.

Declaration
log {
			    source { system(); };
				    if (program("postfix")) {
				      rewrite { set-facility("mail"); };
				    };
				    destination { file("/var/log/mail.log"); };
				    flags(flow-control);
				};
Parameters

The set-facility() rewrite function has a single, mandatory parameter that can be defined as follows:

set-facility( "parameter1" );
Accepted values

The set-facility() rewrite function accepts the following values:

  • numeric strings: [0-7]
  • named values: emerg, emergency, panic, alert, crit, critical, err, error, warning, warn, notice, info, informational, debug
Example usage for the set-facility() rewrite function

The following example can be used in production for the set-facility() rewrite function.

rewrite {
set-facility("info");
set-facility("6");
set-facility("${.json.severity}");};

Was this topic helpful?

[Select Rating]



Setting the priority of a message with the set-pri() rewrite function

You can set the PRI value of a BSD or IETF syslog message with the set-pri() rewrite function by specifying a template string. This is useful, for example, if incoming messages do not have a PRI value specified by default, but a PRI value is required for filtering purposes.

When configured, the set-pri() function will only rewrite the PRI value of the message field.

NOTE: If the specified parameter value is not a valid value, the function ignores it and sends a debug message. However, the syslog-ng Open Source Edition (syslog-ng OSE) application will still send the message.

Declaration
rewrite <rule-name> {
    set-pri("template-string");
};
Parameters

The set-pri() rewrite function expects a template string as its only parameter, for example:

  • set-pri("42");

  • set-pri("$.json.priority");

Accepted values

The template string specified for the set-pri() rewrite function must expand to a natural number in the interval of 0–1023, inclusive. This means that if you, for example, extract the value from a syslog <PRI> header (such as <42>), then you need to remove the opening and closing brackets (< >) in the specified template string.

Example: Temporarily raising the priority of an application

In the following example, the set-pri() rewrite function is used to temporarily raise the priority of the application myprogram:

log {
  source { system(); };
  if (program("myprogram")){
  rewrite { set-pri("92"); };
  };
  destination { file("/var/log/mail.log"); };
  flags(flow-control);
}
Example: Changing the priority of an application log message in JSON format

In the following example, an application sends log messages in the following JSON format:

{
"time": "2003-10-11T22:14:15.003Z",
"host": "mymachine",
"priority": "165",
"message": "An application event log entry."
}

You can parse these logs with the JSON parser function:

{
parser p_json {
json-parser (prefix(".json."));
}

As the application message contains a valid priority field, you can use the set-pri() rewrite function to modify the priority of the message:

set-pri("$.json.priority");

Was this topic helpful?

[Select Rating]



Unsetting message fields

You can unset macros or fields of the message, including any user-defined macros created using parsers (for details, see parser: Parse and segment structured messages and db-parser: Process message content with a pattern database (patterndb)). Note that the unset operation completely deletes any previous value of the field that you apply it on.

NOTE: Hard macros cannot be modified. For details on the hard and soft macros, see Hard versus soft macros).

Use the following syntax:

Declaration:
rewrite <name_of_the_rule> {
    unset(value("<field-name>"));
};
Example: Unsetting a message field

The following example unsets the HOST field of the message.

rewrite r_rewrite_unset{
    unset(value("HOST"));
};

To unset a group of fields, you can use the groupunset() rewrite rule.

Declaration:
rewrite <name_of_the_rule> {
    groupunset(values("<expression-for-field-names>"));
};
Example: Unsetting a group of fields

The following rule clears all SDATA fields:

rewrite r_rewrite_unset_SDATA{
    groupunset(values(".SDATA.*"));
};

Was this topic helpful?

[Select Rating]



Related Documents