Linux system log file

How to View and Configure Linux System Logs on Ubuntu 20.04

This tutorial explains the basic administration of a Linux server through system logs. A system log is a file that contains information about the events that happened on the system during runtime.

In this article, you will learn the following Linux logging basics:

• Where the Linux log files are stored, how are they formatted, and how to read them.
• How to read the most important logs (such as syslog ).
• How to configure the Ubuntu syslog daemon.
• What Linux log rotation is all about and how to use the logrotate utility.

Prerequisites

Before proceeding with the rest of this tutorial, ensure that you have a basic knowledge of working with the Linux command line. While many of the concepts discussed in this article are general applicable to all Linux distributions, we’ll be demonstrating them in Ubuntu only so ensure to set up an Ubuntu 20.04 server that includes a non-root user with sudo access.

🔭 Want to centralize and monitor your Linux logs?

Head over to Logtail and start ingesting your logs in 5 minutes.

Step 1 — Finding Linux system logs

All Ubuntu system logs are stored in the /var/log directory. Change into this directory in the terminal using the command below:

You can view the contents of this directory by issuing the following command:

You should see a similar output to the following:

alternatives.log auth.log btmp cloud-init-output.log dmesg dpkg.log journal/ landscape/ private/ ubuntu-advantage-license-check.log ubuntu-advantage-timer.log unattended-upgrades/ apt/ bootstrap.log cloud-init.log dist-upgrade/ dmesg.0 faillog kern.log lastlog syslog ubuntu-advantage.log ufw.log wtmp 

Let’s look at a few of the essential system log files that may be present in the /var/log directory and what they contain:

• /var/log/syslog : stores general information about any global activity in the system.
• /var/log/auth.log : keeps track of all security-related actions (login, logout, or root user activity).
• /var/log/kern.log : stores information about events originating from the Linux kernel.
• /var/log/boot.log : stores system startup messages.
• /var/log/dmesg : contains messages related to device drivers.
• /var/log/faillog : keeps track of failed logins, which comes in handy when investigating attempted security breaches.

The /var/log directory is also used to store various application logs. For example, if your distribution is bundled with Apache or MySQL, or installed later, their log files will also be found here.

Step 2 — Viewing Linux log file contents

Log files contain a large amount of information that are useful for monitoring or analyzing activities performed by the system or a specific application. Therefore, a Linux server administrator must learn the art of reading and understanding the various messages present in log files to effectively diagnose or troubleshoot an issue.

Before we can read log files, we ought to know how they are formatted. Let’s review two basic approaches to log file formatting and storage: plain text and binary files.

Plaintext log files

These logs are plain text files with a standardized content format. Ubuntu uses a log template called RSYSLOG_TraditionalFileFormat . This log format consists of four main fields with a space delimiter:

1. The timestamp indicates the time when a log entry was created in the format MMM dd HH:mm:ss (e.g. Sep 28 19:00:00 ). Notice that this format does not include a year.
2. Hostname is the host or system that originally create the message.
3. Application is the application that created the message.
4. Message contains the actual details of an event.

Let’s go ahead and review some log files in the plaintext format. Run the command below to print the contents of the /var/log/syslog file with the tail utility:

This outputs the last 10 lines of the file:

Mar 23 12:38:09 peter dbus-daemon[1757]: [session uid=1000 pid=1757] Activating via systemd: service name='org.freedesktop.Tracker1' unit='tracker-store.service' requested by ':1.1' (uid=1000 pid=1754 comm="/usr/libexec/tracker-miner-fs " label="unconfined") Mar 23 12:38:09 peter systemd[1743]: Starting Tracker metadata database store and lookup manager. Mar 23 12:38:09 peter dbus-daemon[1757]: [session uid=1000 pid=1757] Successfully activated service 'org.freedesktop.Tracker1' Mar 23 12:38:09 peter systemd[1743]: Started Tracker metadata database store and lookup manager. Mar 23 12:38:40 peter tracker-store[359847]: OK Mar 23 12:38:40 peter systemd[1743]: tracker-store.service: Succeeded. Mar 23 12:39:01 peter CRON[359873]: (root) CMD ( [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi) Mar 23 12:39:23 peter systemd[1]: Starting Clean php session files. Mar 23 12:39:23 peter systemd[1]: phpsessionclean.service: Succeeded. Mar 23 12:39:23 peter systemd[1]: Finished Clean php session files. 

You’ll notice that that each record in this file is formatted in the manner described earlier. For example, the last record has its timestamp as Mar 23 12:39:23, hostname as peter, application as systemd[1] and message as Finished Clean php session files.

If you want to view the entire log file, you can use the cat utility or any text editor such as nano or vim .

Binary log files

While plaintext is the dominant storage format for log files, you will also encounter binary log files that cannot be read with a normal text editor. The /var/log directory contains multiple binary files that are related to the user authorization:

• /var/log/utmp : tracks users that are currently logged into the system.
• /var/log/wtmp : tracks previously logged in users. It contains a past data from utmp .
• /var/log/btmp : tracks failed login attempts.

For these binary logs, special command-line tools are used to display the relevant information in human-readable form. For example, to review the contents of the /var/log/utmp file, run the who utility with -H option (this option causes column labels to be printed in the output table):

Источник

How to View & Read Linux Log Files

All Linux systems create and store information log files for boot processes, applications, and other events. These files can be a helpful resource for troubleshooting system issues.

Most Linux log files are stored in a plain ASCII text file and are in the /var/log directory and subdirectory. Logs are generated by the Linux system daemon log, syslogd or rsyslogd.

This tutorial will walk you through how to find and read Linux log files, and configure the system logging daemon.

How to View Linux Logs

1. First, open the Linux terminal as a root user. This will enable root privileges.

2. Use the following command to see the log files:

3. To view the logs, type the following command:

The command displays all Linux log files, such as kern.log and boot.log. These files contain the necessary information for the proper function of the operating system.

Log files are accessed using root privileges. By definition, root is the default account that has access to all Linux files.

Use the following example line command to access the respective file:

sudo less [log name here].log

This command displays a timeline of all information related to that operation.

Note that log files are stored in plain text so they can be viewed by using the following standard commands:

zcat – Displays all the contents of logfile.gz

zmore – See the file in pages, without decompressing the files

zgrep – Search inside a compressed file

grep – Find all occurrences of a search term in a file or filter a log file

head – View the very beginning of text files

Note: Check out our comprehensive guide on grep to learn how to use it with examples.

Important Linux System Logs

Logs can tell a lot about the operations of a system. A good understanding of each type of file will help how to distinguish the respective logs.

Most directories can be grouped into one of four categories:

Many of these logs can be located in the var/log subdirectory.

System Logs

Systems log files are needed for Linux to work. On its own, it contains the most significant amount of information about system functionality. The most common log files are:

• /var/log/boot.log : System Boot log (the boot log stores all information related to booting operations)
• /var/log/auth.log : Auth logs (the authentication log stores all authentication logs, including successful and failed attempts)
• /var/log/httpd/ : Apache access and error logs
• /var/log/mysqld.log : MySQL database server log file
• /var/log/debug : Debug logs (the debug log stores detailed messages related to debugging and is useful for troubleshooting specific system operations)
• /var/log/daemon.log : Daemon logs (the daemon log contains information about events related to running the Linux operation)
• /var/log/maillog : Mail server logs (the mail log stores information related to mail servers and archiving emails)
• /var/log/kern.log : Kernel logs (the kernel log stores information from the Ubuntu Linux kernel)
• /var/log/yum.log : Yum command logs

System Logging Daemon

A daemon log is a program that runs in the background and is essential for system operations. These logs have their own category of logs and are seen as the heart of the logging operations for any system.

The path for the system login daemon’s configuration is /etc/syslog.conf .

Each file consists of a selector and an action entry field. The syslogd daemon can forward log messages as well. This can be useful for debugging purposes.

Application Logs

Application logs store information relevant to any application that is executed. This can include error messages, signs of system compromise, and browser identification string.

Log files that fall into this category include CUPS Print System logs, Rootkit Hunter log, Apache HTTP server logs, Samba SMB server logs, and X11 server log.

Non-Human-Readable Logs

Not all logs are designed in a human-readable format. Some are designed only to be read by system applications. Such files are often related to login information. They include login failure logs, last logins logs, and login records.

There are tools and software for reading Linux log files. They are not necessary for reading files as most can be read directly from the Linux terminal.

Supplemental GUIs for Viewing Linux Log Files

System Log Viewer is a GUI that can be used to monitor system logs.

The interface provides several functions for managing logs, including a log statistics display. It is a user-friendly log monitoring GUI.

• A live view of logs
• Number of lines in the log
• Log size
• Most recent log dates
• Modifications made to logs
• Filters
• Keyboard Shortcuts

Alternatively, use Xlogmaster which can monitor a considerable number of log files. It features three different modes:

• Run mode: Starts a specified program and obtains stdout
• Cat mode: Cats files within specified intervals
• Tail mode: Checks log files within regular intervals

Xlogmaster is useful for increasing security. It translates all data for highlighting, hiding lines, and displays this information for taking user requested action.

How to Configure Log Files on Ubuntu and CentOS

This section explains different mechanisms for configuring log files. Let’s start with a CentOS example.

To view users currently logged onto a Linux server, enter the who command as a root user:

This also lists the login history of users. To view the login history of the system administrator, enter the following command:

To view information of the last login, enter:

Execute Log Rotation

Log files that have zeroes appended at the end are rotated files. That means log file names have automatically been changed within the system.

The purpose of log rotation is to compress outdated logs that are taking up space. Log rotation can be done using the logrotate command. This command rotates, compresses, and mails system logs.

logrotate handles systems that create significant amounts of log files. The command is used by the cron scheduler and reads the logrotate configuration file /etc/logrotate.conf . It’s also used to read files in the logrotate configuration directory.

To include additional functionality to logrotate, start by entering the following command:

It compresses and resizes the desired log file.

The commands perform the actions as follows:

missingok – Tells logrotate not to output an error if a log file is missing

notifempty – Does not rotate the log file if it is empty. It reduces the size of the log file with gzip

size – Ensures that the log file does not exceed the specified dimension and rotates it otherwise

daily – Rotates the log files on a daily schedule. This can also be done on a weekly or monthly schedule

create – Instantiates a log file where the owner and group are a root user

A thorough understanding of how to view and read Linux logs is necessary for troubleshooting a Linux system.

Using the proper commands and tools can simplify this process.

Источник