syslog-ng Open Source Edition 3.37 - Administration Guide

Preface Introduction to syslog-ng The concepts of syslog-ng Installing syslog-ng The syslog-ng OSE quick-start guide The syslog-ng OSE configuration file source: Read, receive, and collect log messages
How sources work default-network-drivers: Receive and parse common syslog messages internal: Collecting internal messages file: Collecting messages from text files wildcard-file: Collecting messages from multiple text files kubernetes: Collecting and parsing the Kubernetes CRI (Container Runtime Interface) format linux-audit: Collecting messages from Linux audit logs mqtt: receiving messages from an MQTT broker network: Collecting messages using the RFC3164 protocol (network() driver) nodejs: Receiving JSON messages from nodejs applications mbox: Converting local email messages to log messages osquery: Collect and parse osquery result logs pipe: Collecting messages from named pipes pacct: Collecting process accounting logs on Linux program: Receiving messages from external applications python: writing server-style Python sources python-fetcher: writing fetcher-style Python sources snmptrap: Read Net-SNMP traps sun-streams: Collecting messages on Sun Solaris syslog: Collecting messages using the IETF syslog protocol (syslog() driver) system: Collecting the system-specific log messages of a platform systemd-journal: Collecting messages from the systemd-journal system log storage systemd-syslog: Collecting systemd messages using a socket tcp, tcp6, udp, udp6: Collecting messages from remote hosts using the BSD syslog protocol— OBSOLETE unix-stream, unix-dgram: Collecting messages from UNIX domain sockets stdin: Collecting messages from the standard input stream
destination: Forward, send, and store log messages
amqp: Publishing messages using AMQP collectd: sending metrics to collectd discord: Sending alerts and notifications to Discord elasticsearch2: Sending messages directly to Elasticsearch version 2.0 or higher (DEPRECATED) elasticsearch-http: Sending messages to Elasticsearch HTTP Bulk API file: Storing messages in plain-text files graphite: Sending metrics to Graphite Sending logs to Graylog hdfs: Storing messages on the Hadoop Distributed File System (HDFS) Posting messages over HTTP http: Posting messages over HTTP without Java kafka: Publishing messages to Apache Kafka (Java implementation) kafka-c(): Publishing messages to Apache Kafka using the librdkafka client (C implementation) loggly: Using Loggly logmatic: Using Logmatic.io mongodb(): Storing messages in a MongoDB database mqtt() destination: sending messages from a local network to an MQTT broker network: Sending messages to a remote log server using the RFC3164 protocol (network() driver) osquery: Sending log messages to osquery's syslog table pipe: Sending messages to named pipes program: Sending messages to external applications pseudofile() python: writing custom Python destinations redis: Storing name-value pairs in Redis riemann: Monitoring your data with Riemann slack: Sending alerts and notifications to a Slack channel smtp: Generating SMTP messages (email) from logs snmp: Sending SNMP traps Splunk: Sending log messages to Splunk sql: Storing messages in an SQL database stomp: Publishing messages using STOMP Sumo Logic destinations: sumologic-http() and sumologic-syslog() syslog: Sending messages to a remote logserver using the IETF-syslog protocol syslog-ng(): Forward logs to another syslog-ng node tcp, tcp6, udp, udp6: Sending messages to a remote log server using the legacy BSD-syslog protocol (tcp(), udp() drivers) Telegram: Sending messages to Telegram unix-stream, unix-dgram: Sending messages to UNIX domain sockets usertty: Sending messages to a user terminal: usertty() destination Write your own custom destination in Java or Python Client-side failover
log: Filter and route log messages using log paths, flags, and filters Global options of syslog-ng OSE TLS-encrypted message transfer template and rewrite: Format, modify, and manipulate log messages parser: Parse and segment structured messages
Parsing syslog messages Parsing messages with comma-separated and similar values Parsing key=value pairs JSON parser XML parser Parsing dates and timestamps Python parser Parsing tags Apache access log parser Linux audit parser Cisco parser Parsing enterprise-wide message model (EWMM) messages iptables parser Netskope parser panos-parser(): parsing PAN-OS log messages Sudo parser MariaDB parser Websense parser Fortigate parser Check Point Log Exporter parser Regular expression (regexp) parser db-parser: Process message content with a pattern database (patterndb)
Correlating log messages Enriching log messages with external data Statistics of syslog-ng Multithreading and scaling in syslog-ng OSE Troubleshooting syslog-ng Best practices and examples The syslog-ng manual pages Creative Commons Attribution Non-commercial No Derivatives (by-nc-nd) License Glossary

The syslog-ng manual pages

This chapter collects the manual pages of syslog-ng OSE and other related applications that are usually distributed and packaged together with the syslog-ng Open Source Edition application.


Was this topic helpful?

[Select Rating]



The syslog-ng manual pages

This chapter collects the manual pages of syslog-ng OSE and other related applications that are usually distributed and packaged together with the syslog-ng Open Source Edition application.


Was this topic helpful?

[Select Rating]



dqtool.1

Name

dqtool — Display the contents of a disk-buffer file created with syslog-ng OSE.

Synopsis

dqtool [command] [options]

Description

NOTE: The dqtool application is distributed with the syslog-ng OSE system logging application, and is usually part of the syslog-ng OSE package. The latest version of the syslog-ng OSE application is available at the syslog-ng OSE page.

This manual page is only an abstract, for the complete documentation of syslog-ng OSE, see the syslog-ng OSE Documentation page.

The dqtool application is a utility that can be used to display and format the messages stored in a disk-buffer file.

The cat command

cat [options] [file]

Use the cat command to display the log messages stored in the disk-buffer (also called disk-queue) file, and also information from the header of the disk queue file. The messages are printed to the standard output (stdout), so it is possible to use grep and other tools to find particular log messages, for example, dqtool cat /var/log/messages.lgs |grep 192.168.1.1.

The cat command has the following options:

  • --debug or -d

    Print diagnostic and debugging messages to stderr.

  • --help or -h

    Display a brief help message.

  • --template=<template> or -t

    Format the messages using the specified template.

  • --verbose or -v

    Print verbose messages to stderr.

  • --version or -V

    Display version information.

Example: The cat command
./dqtool cat ../var/syslog-ng-00000.qf

The output looks like:

	Disk-buffer state loaded;
filename='../var/syslog-ng-00000.qf', qout_length='65', qbacklog_length='0', qoverflow_length='9205', qdisk_length='0'
Mar  3 10:52:05 tristram localprg[1234]: seq: 0000011630, runid: 1267609923, stamp: 2010-03-03T10:52:05 PADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADD
Mar  3 10:52:05 tristram localprg[1234]: seq: 0000011631, runid: 1267609923, stamp: 2010-03-03T10:52:05 PADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADD
The relocate command

relocate [options] [files]

Use the relocate command to move or rename disk-buffer (also called disk-queue) files. Note that this option modifies the persist file. Stop syslog-ng OSE before using this command.

The cat command has the following options:

  • --all or -a

    Relocate every disk-buffer file that is listed in the syslog-ng OSE persist file.

  • --new_path or -n

    The directory where you want to move the disk-bufffer files. For example: /var/disk-buffers

  • --persist or -p

    The path to the syslog-ng OSE persist file. The relocate command automatically updates the entries of the disk-buffer files in the persist file.

Examples:

Relocate a single queue file:

bin/dqtool relocate --new_path /tmp/dq --persist var/syslog-ng.persist /tmp/syslog-ng-00000.rqf

Relocate multiple queue files:

bin/dqtool relocate --new_path /tmp/dq --persist var/syslog-ng.persist /tmp/syslog-ng-00000.rqf /tmp/syslog-ng-00001.rqf

Relocate every queue file:

bin/dqtool relocate --new_path /tmp/dq --persist var/syslog-ng.persist --all
Files

dqtool

See also

The syslog-ng.conf manual page

The syslog-ng manual page

NOTE: For the detailed documentation of syslog-ng OSE see syslog-ng OSE Documentation page.

If you experience any problems or need help with syslog-ng OSE, visit the syslog-ng mailing list.

For news and notifications about syslog-ng OSE, visit the syslog-ng blogs.


Was this topic helpful?

[Select Rating]



loggen.1

Name

loggen — Generate syslog messages at a specified rate

Synopsis

loggen [options]

target [port]

Description

NOTE: The loggen application is distributed with the syslog-ng OSE system logging application, and is usually part of the syslog-ng OSE package. The latest version of the syslog-ng OSE application is available at the syslog-ng OSE page.

This manual page is only an abstract, for the complete documentation of syslog-ng OSE, see the syslog-ng OSE Documentation page.

The loggen application is a tool to test and stress-test your syslog server and the connection to the server. It can send syslog messages to the server at a specified rate using a number of connection types and protocols, including TCP, UDP, and unix domain sockets. The messages can be generated automatically (repeating the PADDstring over and over), or read from a file or the standard input.

When loggen finishes sending the messages, it displays the following statistics:

  • average rate: The average rate of the sent messages in messages/second.

  • count: The total number of messages sent.

  • time: The time required to send the messages in seconds.

  • average message size: The average size of the sent messages in bytes.

  • bandwidth: The average bandwidth used for sending the messages in kilobytes/second.

Options
  • --active-connections <number-of-connections>

    Number of connections loggen will use to send messages to the destination. This option is usable only when using TCP or TLS connections to the destination. Default value: 1

    The loggen utility waits until every connection is established before starting to send messages. See also the --idle-connections option.

  • --csv or -C

    Send the statistics of the sent messages to stdout as CSV. This can be used for plotting the message rate.

  • --dgram or -D

    Use datagram socket (UDP or unix-dgram) to send the messages to the target. Requires the --inet option as well.

  • dont-parse or -d

    Do not parse the lines read from the input files, send them as received.

  • --help or -h

    Display a brief help message.

  • --idle-connection <number-of-connections>

    Number of idle connections loggen will establish to the destination. Note that loggen will not send any messages on idle connections, but the connection is kept open using keep-alive messages. This option is usable only when using TCP or TLS connections to the destination. See also the --active-connections option. Default value: 0

  • --inet or -i

    Use the TCP (by default) or UDP (when used together with the --dgram option) protocol to send the messages to the target.

  • --interval <seconds> or -I <seconds>

    The number of seconds loggen will run. Default value: 10

    NOTE: When --interval and --number are used together, loggen will send messages until the period set in --interval expires or the amount of messages set in --number is reached, whichever happens first.

  • --ipv6 or -6

    Specify the destination using its IPv6 address. Note that the destination must have a real IPv6 address.

  • --loop-reading or -l

    Read the file specified in --read-file option in loop: loggen will start reading from the beginning of the file when it reaches the end of the file.

  • --number <number-of-messages> or -n <number-of-messages>

    Number of messages to generate.

    NOTE: When --interval and --number are used together, loggen will send messages until the period set in --interval expires or the amount of messages set in --number is reached, whichever happens first.

  • --no-framing or -F

    Do not use the framing of the IETF-syslog protocol style, even if the --syslog-proto option is set.

  • --quiet or -Q

    Display statistics only when loggen is finished. If not set, the statistics are displayed every second.

  • --permanent or -T

    Keep sending logs indefinitely, without time limit.

  • --rate <message/second> or -r <message/second>

    The number of messages generated per second for every active connection. Default value: 1000

    If you want to change the message rate while loggen is running, send SIGUSR1 to double the message rate, or SIGUSR2 to halve it:

    kill -USR1 <loggen-pid>kill -USR2 <loggen-pid>
  • --read-file <filename> or -R <filename>

    Read the messages from a file and send them to the target. See also the --skip-tokens option.

    Specify - as the input file to read messages from the standard input (stdio). Note that when reading messages from the standard input, loggen can only use a single thread. The -R -parameters must be placed at end of command, like: loggen 127.0.0.1 1061 --read-file -

  • --sdata <data-to-send> or -p <data-to-send>

    Send the argument of the --sdata option as the SDATA part of IETF-syslog (RFC5424 formatted) messages. Use it together with the --syslog-proto option. For example: --sdata "[test name=\"value\"]

  • --size <message-size> or -s <message-size>

    The size of a syslog message in bytes. Default value: 256. Minimum value: 127 bytes, maximum value: 8192 bytes.

  • --skip-tokens <number>

    Skip the specified number of space-separated tokens (words) at the beginning of every line. For example, if the messages in the file look like foo bar message, --skip-tokens 2 skips the foo bar part of the line, and sends only the message part. Works only when used together with the --read-file parameter. Default value: 0

  • --stream or -S

    Use a stream socket (TCP or unix-stream) to send the messages to the target.

  • --syslog-proto or -P

    Use the new IETF-syslog message format as specified in RFC5424. By default, loggen uses the legacy BSD-syslog message format (as described in RFC3164). See also the --no-framing option.

  • --unix </path/to/socket> or -x </path/to/socket>

    Use a UNIX domain socket to send the messages to the target.

  • --use-ssl or -U

    Use an SSL-encrypted channel to send the messages to the target. Note that it is not possible to check the certificate of the target, or to perform mutual authentication.

  • --version or -V

    Display version number of syslog-ng.

Examples

The following command generates 100 messages per second for ten minutes, and sends them to port 2010 of the localhost via TCP. Each message is 300 bytes long.

loggen --size 300 --rate 100 --interval 600 127.0.0.1 2010

The following command is similar to the one above, but uses the UDP protocol.

loggen --inet --dgram --size 300 --rate 100 --interval 600 127.0.0.1 2010

Send a single message on TCP6 to the ::1 IPv6 address, port 1061:

loggen --ipv6 --number 1 ::1 1061

Send a single message on UDP6 to the ::1 IPv6 address, port 1061:

loggen --ipv6 --dgram --number 1 ::1 1061

Send a single message using a unix domain-socket:

loggen --unix --stream --number 1 </path/to/socket>

Read messages from the standard input (stdio) and send them to the localhost:

loggen 127.0.0.1 1061 --read-file -
Files

/opt/syslog-ng/bin/loggen

See also

The syslog-ng.conf manual page

NOTE: For the detailed documentation of syslog-ng OSE see syslog-ng OSE Documentation page.

If you experience any problems or need help with syslog-ng OSE, visit the syslog-ng mailing list.

For news and notifications about syslog-ng OSE, visit the syslog-ng blogs.


Was this topic helpful?

[Select Rating]



Related Documents