Work with system logging on Oracle Linux
In this tutorial, you configure system logging, use rsyslog templates to format log messages, install and run logwatch , view the journald journal , and configure persistent journald storage.
Background
System log files contain messages about the system, kernel, services, and applications. The journald logging daemon, which is part of systemd , records system messages in non-persistent journal logs in memory and in the /run/log/journal directory. journald forwards messages to the system logging daemon, rsyslog . As files in /run are volatile, the log data is lost after a reboot unless you create the directory /var/log/journal . You can use the journalctl command to query the journal logs.
Objectives
- Explore the rsylog.conf file
- Explore the logrotate.conf file
- Configure and use rsyslog templates
- Install logwatch and run the logwatch utility
- Explore and use journald
What Do You Need?
A fully patched Oracle Linux 8 or later system.
Note: When using the free lab environment, see Oracle Linux Lab Basics for connection and other usage instructions.
Explore and Modify the rsyslog.conf File
Oracle Linux 8 relies on the service, rsyslogd , to facilitate logging. This daemon permits the processing and forwarding of logs on your system. /etc/rsyslog.conf file is the main configuration file for system logging.
Watch the video below for an overview on rsyslog .
- The MODULES section uses the $ModLoad directive to load the modules.
- The GLOBAL DIRECTIVES section specifies configuration options.
- The RULES section defines a facility ( facility.priority ) and an action.
Press the q key to exit the less command.
- Enter insert mode by typing i .
- On the next line, change /var/log/cron to /var/log/cron_new , as shown:
systemctl restart rsyslog Type ESC , :wq! and ENTER to save and close the file.
systemctl restart rsyslog systemctl restart rsyslog logger -p info "This is an info-priority message" logger -p debug "This is an debug-priority message" Explore and Modify Log Rotation
Oracle linux installs the logrotate utility to rotate logs. logrotate places a logrotate.conf configuration file the /etc directory at install time. The logwatch configuration file contains the directives for the default rotation frequency. You can modify the frequency of the log rotation by editing this configuration file.
- Use the ls command to view represtations of rotated logs in the /var/log directory.
- In a production system, files with numbers at the end of the file name represent rotated logs with time stamps added to the log file names.
- The rotated log files provide a history of events, and the files without time stamps provide a record of the most current events.
- The number of rotated files depend on the how long the system has been running.
cat /etc/logrotate.d/firewalld - missingok : If the log file is missing, do not issue an error message.
- nocreate : New log files are not created.
- postrotate / endscript :The lines between these directives are executed after the log file is rotated.
- sharedscripts :The postrotatescript runs only once, not once for each log that is rotated.
Note: For a full list of directives and configuration options, refer to the logrotate(8) man page.
Use rsyslog Templates
Templates allow you to specify and format rsyslogd output the way a user might want. A template consists of a template directive, a name, the actual template text and optional options.
$template TEMPLATE_NAME,"text %PROPERTY% text", [OPTION] - $template : Directive that notifies rsyslog this line is a template
- TEMPLATE_NAME : Defines the name of the template
- «text» : Actual template text surrounded by quotation marks
- %PROPERTY% : Specific message content surrounded by percent signs
- OPTION : Specifies options that modify the template functionality
- Use the command vi /etc/rsyslog.conf to define a template.
- Add the template definition line shown in the code box to the bottom of the file.
- This entry creates a template named lab .
- Do not exit the vi editor.
- Add a new line after the template definition line.
- This entry writes all messages to the /var/log/lab.log file and formats the entries by using the log template.
- Exit the vi editor and save the file after adding the new line.
- Enter the following:
systemctl restart rsyslog The lab.log file contains all entries preceded by the text “ Message: ” followed by the actual message, as defined in the lab template.
$template lab, "Time: %timestamp%, Facility: %syslogfacility-text%, Priority: %syslogpriority-text%, Hostname: %hostname%, Message: %msg%\n" systemctl restart rsyslog The newest entries now include the Time , Facility , Priority , Hostname , and Message properties, as defined in the class template.
Install and Use logwatch
logwatch is a customizable log monitoring system. It analyzes system logs for a given time period and reports on specific areas of interest.
It might be necessary to install the logwatch packages. After installing it, logwatch is configured by default to run each night as defined in /etc/cron.daily/0logwatch and email a report to the root user.
Watch the video below for an overview on logwatch .
The output lists the logwatch packages as Available Pachages only, which means you need to install the utiltiy. The packages are availabe in the ol8_baseos_latest repository located in the /etc/yum.repos.d/oracle-linux-ol8.repo file. This repository is enabled by default.
- The main configuration file is /usr/share/logwatch/default.conf/logwatch.conf .
- Local configuration options can be set in /etc/logwatch/conf/logwatch.conf .
- A HOWTO-Customize-Logwatchfile file exists in the /usr/share/doc/logwatch/ directory. This file describes the structure of the Logwatch files in this version, how to modify the configuration files for your system, and how to create new service filters.
less /usr/share/logwatch/default.conf/logwatch.conf - LogDir : Is the default log directory, and all files are placed relative to this directory
- TmpDir : Is a temporary directory you define to override the default /tmp directory
- MailTo : Is the user to mail the reports to. root is the default recipient.
- MailFrom : Is the default person to mail reports from. Logwatch is the default sender.
- Range : Is the default time range for the report. yesterday is the default range .
- Detail : Is the default detail level for the report. Detail levels can be set to: Low or O , Med or 5 , or High or 10 . Low is the default detail level.
- Service : Is the default services to report on as defined in /usr/share/logwatch/scripts/services/ . All services is the default. You can also disable certain services even when specifying All .
less /etc/cron.daily/0logwatch ################### Logwatch 7.4.3 (04/27/16) #################### Processing Initiated: Thu Oct 27 14:55:54 2021 Date Range Processed: today ( 2021-Oct-27 ) Period is day. Detail Level of Output: 0 Type of Output/Format: stdout / text Logfiles for Host: ol8-server ################################################################## Use journald
Log files can also be managed by the journald daemon, which is part of systemd . The full service name is systemd-journald.service , and the full daemon name is systemd-journald .
You use journalctl to query the systemd journal logs. By default, the listed entries include a time stamp, the host name, the application involved, and the given message.
The main configuration file for systemd-journald is /etc/systemd/journald.conf .
In this practice, you use the journalctl command to query the systemd journal, view journald metadata, and enable persistent journald storage. Your log output might vary.
Watch the video below for an overview on journald and journalctl .
- Use the journalctl command with no options or arguments to query the systemd journal.
- Enter q to exit journalctl .
- All log data is displayed, including rotated logs.
- The beginning of the boot process is indicated with a special entry.
- Entries with error priority and higher are in red.
- Entries with notice and warning priority are in bold font (which might not be visible in your terminal window; opening a terminal window in the GNOME GUI will show the bold font).