Linux web server log

How to View and Configure Apache Access & Error Logs

At the time of writing, the Apache HTTP server is used by 30.8% of all web servers in operation. If you’re responsible for managing any system that utilizes Apache, then you will surely interact with its logging infrastructure on a regular basis. This tutorial will introduce you to logging in Apache and how it can help you diagnose, troubleshoot, and quickly resolve any problem you may encounter on your server.

You will learn where logs are stored, how to access them, and how to customize the log output and location to fit your needs. You will also learn how to centralize Apache logs in a log management system for easier tracing, searching, and filtering of logs across your entire stack.

🔭 Want to centralize and monitor your Apache logs?

Head over to Logtail and start ingesting your logs in 5 minutes.

Prerequisites

To follow through with this tutorial, you should set up a Linux server that includes a non-root user with sudo privileges. Additionally, you also need the Apache HTTP server installed and enabled on the server, which can be done by executing the relevant commands below.

On Debian-based distributions like Ubuntu:

sudo systemctl enable apache2 

sudo systemctl start apache2 

sudo systemctl enable httpd 

sudo systemctl start httpd 

Please note that the rest of the commands, directory configurations, and conventions used in this tutorial pertain to Debian-based distributions like Ubuntu. Still, the concepts remain the same for other distributions.

Step 1 — Getting started with Apache logging

Apache logs are files that record everything the Apache web server is doing for later analysis by the server administrator. The records of all Apache events are placed in two different text files:

• Access Log: this file stores information about incoming requests. You’ll find details about each request such as the requested resource, response codes, time taken to generate the response, IP address of the client, and more.
• Error Log: this file contains diagnostic information about any errors were encountered while processing requests.

Step 2 — Locating the Apache log files

The log files’ location depends on the operating system the Apache web server is running. On Debian-based operating systems like Ubuntu, the access log file is located in /var/log/apache2/access.log . On CentOS, RHEL, or Fedora, the access log file is stored in /var/log/httpd/access_log .

A typical access log entry might look like this:

::1 - - [13/Nov/2020:11:32:22 +0100] "GET / HTTP/1.1" 200 327 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36" 

Similarly, the error log file is located in /var/log/apache2/error.log on Debian-based systems and /var/log/httpd/error_log on CentOS, RHEL, or Fedora. A typical error log entry might look like this:

[Thu May 06 12:03:28.470305 2021] [php7:error] [pid 731] [client ::1:51092] script '/var/www/html/missing.php' not found or unable to stat 

In the next section, we’ll discuss how to view these log files from the command line.

Step 3 — Viewing Apache Log files

One of the most common ways to view an Apache log file is through the tail command which prints the last 10 lines from a file. When the -f option is supplied, the command will watch the file and output its contents in real-time.

sudo tail -f /var/log/apache2/access.log 

You should observe the following output on the screen:

. . . 198.54.132.137 - - [04/Feb/2022:11:34:04 +0000] "GET / HTTP/1.1" 200 3477 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36" 198.54.132.137 - - [04/Feb/2022:11:34:04 +0000] "GET / HTTP/1.1" 200 3477 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36" 198.54.132.137 - - [04/Feb/2022:11:34:04 +0000] "GET / HTTP/1.1" 200 3477 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36" 198.54.132.137 - - [04/Feb/2022:11:34:05 +0000] "GET / HTTP/1.1" 200 3476 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36" 198.54.132.137 - - [04/Feb/2022:11:34:06 +0000] "GET / HTTP/1.1" 200 3476 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36" 198.54.132.137 - - [04/Feb/2022:11:34:06 +0000] "GET / HTTP/1.1" 200 3476 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36" 198.54.132.137 - - [04/Feb/2022:11:34:07 +0000] "GET / HTTP/1.1" 200 3476 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36" 198.54.132.137 - - [04/Feb/2022:11:34:07 +0000] "GET / HTTP/1.1" 200 3476 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36" 

To view the entire contents of the file, you can use the cat command or open the file in a text editor like nano or vim :

cat /var/log/apache2/access.log 

You may also want to filter the log entries in the log file by a specific term. In such cases, you should use the grep command. The first argument to grep is the term you want to search for, while the second is the log file that will be searched. In example below, we are filtering all the lines that contain the word GET :

sudo grep GET /var/log/apache2/access.log 

This should present the following output:

. . . 198.54.132.137 - - [04/Feb/2022:11:34:04 +0000] "GET / HTTP/1.1" 200 3477 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36" 198.54.132.137 - - [04/Feb/2022:11:34:04 +0000] "GET / HTTP/1.1" 200 3477 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36" 198.54.132.137 - - [04/Feb/2022:11:34:05 +0000] "GET / HTTP/1.1" 200 3476 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36" 198.54.132.137 - - [04/Feb/2022:11:34:06 +0000] "GET / HTTP/1.1" 200 3476 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36" 198.54.132.137 - - [04/Feb/2022:11:34:06 +0000] "GET / HTTP/1.1" 200 3476 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36" 198.54.132.137 - - [04/Feb/2022:11:34:07 +0000] "GET / HTTP/1.1" 200 3476 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36" 198.54.132.137 - - [04/Feb/2022:11:34:07 +0000] "GET / HTTP/1.1" 200 3476 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36" 

Step 4 — Examining Apache access log formats

The access log records all requests that are processed by the server. You can see what resources are being requested, the status of each request, and how long it took to process their response. In this section, we’ll dive deeper into how to customize the information that is displayed in this file.

Before you can derive value from reading a log file, you need to understand the format that is being used for each of its entries. The CustomLog directive is what controls the location and format of the Apache access log file. This directive can be placed in the server configuration file ( /etc/apache2/apache2.conf ) or in your virtual host entry. Note that defining the same CustomLog directive in both files may cause problems.

Let’s look at the common formats used in Apache access logs and what they mean.

Common Log Format

The Common Log Format is the standardized access log format format used by many web servers because it is easy to read and understand. It is defined in the /etc/apache2/apache2.conf configuration file through the LogFormat directive.

When you run the command below:

Источник

How do I view Nginx logs?

Logs are very important in a system to monitor the activities of an application as they provide you with useful debugging information and enable you to analyze all aspects of a web server. Like the other software applications, Nginx also maintains events like your web site visitors, encountered problems, and more to log files. The useful recorded information is used to take preemptive measures in order to deal with major serious discrepancies in the log events.

In this article, we will elaborate on how to configure and view Nginx Logs in Ubuntu 20.04 system to monitor the application activities.

There are two types of logs where recorded events in Nginx one is the access log, and the other is the error log. If you have already enabled these logs in the Nginx core configuration file then, you can find both types of logs in /var/log/nginx in all Linux distributions.

Nginx Access log

All activities related to site visitors are recorded in the access logs. In this type of log, you can find those files which are recently accessed, how the Nginx responded to a client request, client IP addresses, what browser a client is using, and more. By using the information of the access log, you can monitor the traffic to find site usage over time. If you monitor the access logs properly, then you can easily find some unusual requests which are sent by a user to check the flaws in the deployed application.

Enable the Nginx Access log

The access log you can enable with the access_log directive either in the server section or in HTTP.

The first argument, ‘log_file’ is compulsory, whereas the second argument is optional, ‘log_format’. If you do not mention log format, then logs will be typed in the default combined format.

The access log is defined by default in the Nginx configuration file. So, all virtual host’s access logs will be stored in the same configuration file.

It is recommended to set apart the access logs of all virtual hosts by recording into a new separate file.

Reload the new NGINX configurations. Now, you can visit the access logs for the example.com domain in the file /var/log/nginx/example.access.log, by using the following command:

Customize format in the Access log

Let’s explain an example to define a custom access log format. By default, the access log is recorded in a combined log format. Therefore, you can extend the predefined format with the value of gzip response for compression ratio.

http {
log_format custom ‘$remote_addr — $remote_user [$time_local] ‘
‘»$request» $status $body_bytes_sent ‘
‘»$http_referer» «$http_user_agent» «$gzip_ratio»‘ ;

Once you have made all changes in the configuration of Nginx, reload the Nginx and then run the tail command to display the gzip ratio at the end of the event log.

NGINX error log

If NGINX is suddenly stopped running or not working properly, it will record all events in the error log. Therefore, using the error logs, you can find more details. It also records warnings, but it cannot identify a problem that has occurred.

Enable error log

The following syntax of error_log directive:

In the above syntax, the first argument represents the log file path, and the second argument identifies the security level of the log event.

We have mentioned an example below in which performing overriding in error_log directive in the server context.

When you are required to disable the error log, assign the name of the log file to /dev/null.

Nginx Security Level of Error log

The following security level you can use in the error log:

1. emerg: When your system is unstable, used for emergency messages
2. alert: Generate alert messages of serious problems.
3. crit: Used for Critical issues for immediately dealing.
4. error: While processing a page, an error may occur.
5. warn: Used for a warning message
6. notice: Notice log that you can also ignore.
7. info: For information, messages
8. debug: Points the error location used for debugging information.

Conclusion

Nginx access and error logs are useful for recording certain activities. We have learned how we can enable and view these types of Nginx logs on our Linux system. That’s all about the Nginx logs.

About the author

Karim Buzdar

Karim Buzdar holds a degree in telecommunication engineering and holds several sysadmin certifications. As an IT engineer and technical author, he writes for various web sites. He blogs at LinuxWays.

Источник