Search:  
Gentoo Wiki

Rsyslog


This article is part of the HOWTO series.
Installation Kernel & Hardware Networks Portage Software System X Server Gaming Non-x86 Emulators Misc
Warning: This HOWTO is a work in progress! I only include examples I've tested though.

Contents

Introduction

Rsyslog is yet another system logger. It supports hugely diverse methods of filtering and many output methods including being able to manipulate the output format. It can log to pipes, files databases and so on. It supports TCP and UDP and RELP which is a highly optimized and reliable protocol designed for logging. It is fully backwards compatible with syslog, so you can give it a "traditional" configuration and it will just work, providing an easy upgrade path. It even creates the necessary file structure as needed. The level of customization is huge and possibly intimidating at first (the documentation is complete on the project website but not the easiest to follow!)

Other system loggers are available. On a personal note, I have used syslog-ng for some years for many systems. However the database output is available only in the "commercial" version (yes there are ways to do it via pipes) as is SSL encryption. The commercial side of rsyslog however is related only to integration with Windows based systems. I don't begrudge people making a living out of their software but in the end, rsyslog offers the whole package for my needs (and then some). Frankly it is clear that this gentleman: http://blog.gerhards.net/ is completely fixated with system loggers and has a serious amount of experience with them. Several other distros are moving to use rsyslog as the default (Fedora already and Debian soon). When I first started worrying about logging I slavishly followed the "standard" way of doing things with the facility:priority thing. Syslog-ng showed that this was not necessary and now rsyslog seems able to remove all barriers to what can be done with a system logger.

In this howto I will present an easily extensible framework for configuration. The key is that you can include directories of mini config files just like logrotate etc. This feature would make it easy to add logging to rsyslog via a USE flag for each package that supports using a system logger. For now though whenever you add something that needs syslog, you just add a new mini config and reload the daemon.

Installing

At the time of writing (rsyslog is still under heavy development) the following USE flags are available. Enable mysql or postgresql for database logging. RELP is a protocol for logging that currently on rsyslog supports. SNMP means that rsyslog can send traps based on log entries. GNUTLS is for secure logging.

kerberos mysql postgres snmp zlib dbi debug gnutls relp

rsyslog is marked ~x86 so you may need something like this (check the main Gentoo documentation for details on what this means):

#echo "=app-admin/rsyslog-3.21.2 ~x86" >> /etc/portage/package.keywords
#emerge -va rsyslog

Configuration Examples

The ebuild installs a basic /etc/rsyslog.conf file. Here is the one I use (note the IncludeConfig):

Code: Code /etc/rsyslog.conf
# /etc/rsyslog.conf
# RSyslog system logger configuration
# Docs: http://www.rsyslog.com/doc-rsyslog_conf.html
# "/etc/init.d/rsyslog reload" to reread the config
# Jon Gerdes (www.blueloop.net) 23 Aug 2008
#
# JG 24 Aug 2008
# - Extra spaces for legibility in params causes errors, OK in
#   log output lines

# Modules --------------------------------------------------------------------
# Input
$ModLoad immark.so      # Add Mark messages
$ModLoad imuxsock.so    # Unix sockets
$ModLoad imklog.so      # Kernel logger
#$ModLoad imudp.so       # UDP input
#$ModLoad imrelp.so      # RELP input
#$ModLoad imtcp.so       # TCP input
#$ModLoad file.so        # Text file input
#$ModLoad imgssapi.so    # Plain TCP and GSSAPI
#$ModLoad im1395.so      # Messages via RFC1395

# Output
#$ModLoad omsnmp.so      # Send SNMP traps
#$ModLoad ommysql.so     # Log to MySQL
#$ModLoad ompgsql.so     # Log to PostgreSQL
#$ModLoad ommail.so      # Send mail
#$ModLoad omrelp.so      # Send to another host via RELP
#$ModLoad omlibdbi.so    # Log via generic DB output
#$ModLoad omgss.so       # GSS enabled output

# Globals --------------------------------------------------------------------
# There are many more - see docs
# Files and dirs are created as needed (dirs only for "dynamic" files)
$umask 0000
$DirCreateMode 0640
$FileCreateMode 0640
$FileOwner rsyslog
$FileGroup rsyslog
$DirOwner rsyslog
$DirGroup rsyslog
$RepeatedMsgReduction on

# Listeners ------------------------------------------------------------------
$UDPServerRun 514                      # UDP receiver
#$InputTCPServerRun 514                 # TCP receiver
#$InputRELPServerRun 20514              # RELP receiver
#$InputGSSServerRun 1514                # Kerberos
#$InputGSSServerPermitPlainTCP on       # Add plain text listener to above

# Logging --------------------------------------------------------------------

# Log remotely (check docs for additional options, eg port and compression)
#*.*        @192.168.100.100     # UDP
#*.*       @@logserver.my.domain # TCP

# Filter out unwanted messages
#wpa_supplicread_interface: No such device

# Each ebuild that honours USE=rsyslog will have a snippet
#   added to /etc/rsyslog.d/
# These will log and then filter out those entries so that
#   they are not repeated in later outputs.
# Include package specific logs (including rsyslog itself)

$IncludeConfig /etc/rsyslog.d/*.conf

# syslog style - everything else that falls through from above
# will be logged in "traditional syslog style"
# If you don't want package specific logs, comment out the include above
# The "-" means don't sync after each write
# & ~ means drop messages that the last filter found (avoid duplication)

# Log to the console
kern.*                                          /dev/console
&                                               /var/log/kernel
& ~

*.info;mail.none;authpriv.none;cron.none        -/var/log/messages
& ~

authpriv.*                                      /var/log/secure
& ~

mail.*                                          -/var/log/maillog
& ~

cron.*                                          -/var/log/cron
& ~

# Display using wall to all logged in users
*.emerg                                         *
& ~

uucp,news.crit                                  -/var/log/spooler
& ~

local7.*                                        /var/log/boot.log
& ~

# Finally log everything else
*.*                                             /var/log/uncategorized.log

Warning: The config above needs some work. I want to ensure that all messages end up somewhere or are deliberately dropped

I have added a user and group called rsyslog which are set as the owner and group of created files and directories. I've commented out the UDP listener in the config above to avoid a copy n paste without reading it causing a problem for someone.

In /etc/rsyslog.d/ there are a series of files (<anything>.conf will be included)

Here are some examples. Put each one into a separate file for example "/etc/rsyslog.d/exim.conf". Whilst testing, just use "/etc/init.d/rsyslog reload" to reread and use the new configuration.

I have set Exim to use syslog as its logger:

# exim
:programname, isequal, "exim" /var/log/mail/mail.log
& ~

The : indicates a filter statement with an action. If the programname field in a received log message is equal to "exim" then send the entry to /var/log/mail/mail.log. The second line does the same filter (the & repeates the previous filter) but sends it to the "~" (tilde) action which is discard. Without the second statement, the entry would also get logged by the "mail.*" entry further down in the main rsyslog.conf file. Using the & format saves having to perform the filter twice which is more efficient and helps avoid typos.

# Firewall Builder
:msg, contains, "FWB-RULE" /var/log/firewall.log
& ~

Now we are filtering on the message body. I have used FWBuilder and told it to log with FWB-RULE as a prefix. After messing about with ulogd etc this makes for a really powerful firewall logger.

# ntpd
:programname, isequal, "ntpd" /var/log/ntpd.log
& /dev/tty12
& ~

You can stack several actions in a chain, here the ntpd's log also appears on tty12 on the console (ALT-F12 or CTRL-ALT-F12 in X)

# drayteks
$template TDraytek,"/var/log/draytek/%HOSTNAME%-%SYSLOGFACILITY-TEXT%-%SYSLOGSEVERITY-TEXT%.log"
if $source contains 'draytek' then ?TDraytek
& ~

Here I am logging the output from my Draytek ADSL routers using the hostname field (note the $UDPServerRun in rsyslog.conf to enable a UDP listener). This example uses "Rainer script" format. Apparently the syntax may change so be careful when updating to a new version that this will still work. First you define a template which I have called TDraytek, then if the source (you can use hostname as well) contains the string draytek, then use the template to log to a dynamically created file (the directory will be created as well). There are options to set the owner and groups. See the globals section in the config. This is powerful stuff, by changing the template, you can split up the output into multiple files which makes reading them easier without digging out grep!

Log rotation

The Output Channel mechanism allows for log rotation, meaning that you could dispense with the logrotate system or call it using rsyslog as needed by running logrotate with a config specific to the log that has reached its specified size. Rsyslog is in development and the docs warn that the Output Channel's syntax will probably change so I wont cover it here yet. Just add each of the files that rsyslog is logging to the /etc/logrotate.d/rsyslog.conf file.

Further reading

http://www.rsyslog.com/ - Website for the project

http://www.rsyslog.com/doc-rsyslog_conf.html - The main rsync.conf docs

http://www.rsyslog.com/doc-property_replacer.html - Notes on the properties that can be filtered on

http://freshmeat.net/articles/view/2809/ - Freshmeat article written by Rainer Gerhards about logging to MySQL

They have a wiki as well.

Other stuff

BIND - named

Add something like this to named.conf (it is its own section and not part of options"):

logging {

   channel "syslog_channel" {
       syslog     daemon;
       severity   info;
   };

   category "queries"  { "syslog_channel"; };
   category "xfer-in"  { "syslog_channel"; };
   category "xfer-out" { "syslog_channel"; };
   category "config"   { "syslog_channel"; };
   category "resolver" { "syslog_channel"; };
   category "update"   { "syslog_channel"; };
   category "default"  { "syslog_channel"; };

};

You could then use filters to split out the various bits into separate files or send all bar queries to files and queries to a database.

Retrieved from "http://www.gentoo-wiki.info/Rsyslog"

Last modified: Thu, 02 Oct 2008 13:23:00 +0000 Hits: 1,151