In this lesson, you will learn about the syslog standard, facilities and severities, syslog servers, logging rules, how to determine what files are used for which messages, how to generate your own log messages, and how to automatically rotate log files. Linux uses the syslog standard for message logging. This allows programs and applications to generate messages that can be captured, processed and stored by a system logger. It eliminates the need for each and every application having to implement a logging mechanism. It also means that logging can be configured and controlled in a central location.

The syslog standard uses facilities and severities to categorize messages. Each message is labeled with a facility code and a severity level. The various combinations of facilities and severities can be used to determine how a message is handled. Facilities are used to indicate what type of program or what part of the system the message originated from. For example, messages that are labeled with a current facility originate from the Linux kernel. Messages that are labeled with a mail facility, come from applications involved in handling mail. Each facility has a number and a keyword associated with it.

Notice the local facilities. They are local zero through local seven. You can use these for your own custom applications. This table lists each of the severities including their code, keyword, and description. The emergency severity is the most severe and debug is the least severe. A syslog server accepts syslog messages and processes those messages based on a set of configurable rules. Traditionally, the syslogd Daemon filled this role but many Linux distributions ship with alternatives such as our rsyslog and syslog-ng. For the remainder of this lesson, we're going to focus on rsyslog. However, the concepts apply to any syslog server.

The main configuration file for rsyslog is /etc/rsyslog.conf. You can include additional configuration files by using the IncludeConfig directive. For example, this line will cause rsyslog to read and include all configuration files that end in .conf, and that also exist in the /etc/rsyslog.d directory. Logging rules consist of two fields. The first field is called the selector field and it lists the facilities and severities of messages to include in the rule. The second field is called the action field. And it determines what will happen to the messages matched by the selector field.

The most common action is to write the messages to a log file. The selector field and action field can be separated by one or more spaces or tabs. The format of the selector field is FACILITY.SEVERITY. Note that wildcards are supported. For example, to match all the mail messages, use mail.star. You can also admit the dot severity if you want to include all messages from the facility. So, mail.star and mail are equivalent. If you do not want to match any messages from a facility, use FACILITY.none. If you want to match multiple facilities severity pairs, separate each pair with a semi-colon. This example rule matches messages that have the facility of mail and any severity. It writes all the matching messages to /var/log/mail.log.

If the path starts with a minus sign that tells syslog that it doesn't have to perform a sync operation for each log message. This is sometimes called caching mode. When using caching mode, know that some messages might be lost if a system crashes immediately after a write attempt. However, you may see performance improvements during normal operations if you have a system that performs many logging operations. The default configurations that ship with a Linux distribution will probably have a mix of caching and non caching rules, with less critical messages using caching.

In this example, there are separate actions for different severities of mail messages. Notice that the less critical mail messages are using caching mode. This example is taken from an Ubuntu system. The first line ensures all messages from the auth and authpriv facilities are written to /var/log/auth.log. The second line writes all messages except ones originating from the auth and authpriv facilities to /var/log/syslog. This particular example is taken from a Red Hat Enterprise Linux system. The rule tells rsyslog to write all messages except mail, authpriv, and cron to /var/log/messages.

Hopefully you can start to see why it's important to understand how to determine where messages are being sent, instead of simply memorizing that system messages are stored in /var/log/messages. Different distributions ship with different configurations and different companies, organizations, and system administrators, may alter the default rules to suit their particular needs. You can use the logger command to generate syslog messages. This can prove to be useful if you want to test any configuration changes you've made to the system logger, or if you want to generate log messages from your own shell scripts.

Use the -p option to provide a FACILITY.SEVERITY level. If you don't specify a facility severity pair, it will default to user.notice. You can also use the -t option to tag your message. In this is example, we generate a message with a mail facility at a severity level of info. You can see that the message made its way to the proper log file. You can use the log rotate tool to rotate, compress, remove, and even mail log files. This provides an automated way to manage log files and can help prevent filling up your storage space with log messages. The configuration file for log rotate is located at 'etc/logrotate.com.

Like many other configuration files, it may use an include directive. This line tells logrotate to read configuration files located in the /etc/logrotate.d directory. Here's a sample logrotate.com file. The configuration in the main file contains some defaults. For example, the weekly keyword ensures that log files will be rotated every week. The rotate Four line tells a logrotate to keep four weeks' worth of logs. Logs older than this will be removed. The create line makes sure that a new empty log file is created after it is rotated. Here's the sample log rotate configuration file from an Ubuntu system. It is located at /etc/logrotate.d/rsyslog. It handles the log rotation for all files associated with rsyslog.

Notice that the format is a single log file, or a list of log files, and followed by the configuration that controls those log files. The configuration is enclosed in brackets. Here's a look at the configuration options that were used in this particular configuration file. Rotate. Rotate the files by count times before removing them. The weekly keyword needs to rotate log files weekly. Missingok means to ignore missing log files. Notifempty indicates to logrotate to not rotate the log file if it's empty. Compress means to compress the rotated log files. Postrotate. The lines between postrotate and endscript are executed using binsh. These commands are executed after the rotation. If you make changes to your logrotate configuration and want to test it, use the following command. Logrotate -fv followed by the configuration file. In this case, /etc/logrotate.conf. The -f option tells log rotate to force a rotation, while the -v option enables verbose logging.

In this lesson you learned about the syslog standard and how it assigns a facility and severity to each message. You also learn that syslog servers employ the use of logging rules to determine what action to perform on a given message. Typically, the action is to simply store the message in a log file. You also learned how to test the syslog server configuration by generating messages with a logger utility. Finally, you learn how to use logrotate to automatically prune system logs.

System Logging