POPSuL:
Сегодня мы поговорим о том что такое ротация логов, с чем ее едят и насколько она вкусна. Ротация — процесс обновления чего‐то , круговорот чего‐то . Но что такое ротация логов?
Ротация логов — это процесс архивирования логов, удаление старых логов и все из этого вытекающее. Оно нужно для того, чтобы внезапно у вас не забилось все место на HDD/SSD различными логами, ну, а так же для того, чтобы не было слишком много одинаковых и очень больших лог‐файлов . При ротации логов происходит несколько вещей:
- Файл журнала архивируется и сохраняется под именем log.log.0
- Более старые логи переименовываются (log, log,0 в log.log.1 и т.д.)
- Самые старые логи удаляются
- Очищается «главный» файл лога
- Перезапускается демон пишуший логи если это необходимо
Порядок всех этих действий может быть произвольным, и зависит от напрямую от реализации утилиты для ротации логов. В linux‐based системах зачастую используется утилита logrotate которая запускается по умолчанию (покрайней мере в ubuntu) кроном раз в день и делает свою грязную работу. В bsd системах, насколько мне известно, используется newsyslog для той же самой работы, но т.к. freebsd я видел всего несколько раз в жизни — ничего полезного я сказать о ней не могу :).
Logrotate имеет свой конфигурационный файл /etc/logrotate.conf в котором описаны глобальные параметры, а так же, подключаются различные дополнительные конфиги из /etc/logrotate.d в которых описаны действия для различных демонов. Например, конфиг nginx выглядит примерно так:
Конфигурационные файлы logrotate человекопонятные. В частности, в этом конфигурационном файле описываются правила для файлов попадающийх под glob‐шаблон /var/log/nginx/*.log:
- missingok — говорит о том, что если файла нету то все ок
- daily — проводить ротацию ежедневно
- rotate X — хранить X файлов
- compress — сжимать старые файлы
- delaycompress — не сжимать последний лог
- notifempty — не архивировать пустые файлы
- create ACCESS_MODE USER GROUP — с какими правами создавать архивы
- sharedscripts — какая то фишка для prerotate и postrotate:)
- prerotate, postrotate — что сделать перед ротацией и после нее.
Что меня побудило написать эту статью? Написать эту статью я решил потому что обнаружил то, что на моих серверах которые я админю логи apache и nginx внезапно стали помещаться в домашнюю директорию пользователя, и имели различные размеры (от пары мегабайт до нескольких гигов), и конечно же ротация логов не была перенастроена, и так вот логами было съедено приличный объем дискового пространства.
Мораль сей басни такова: работает — не трогай, блядь, и все будет работать отлично! А если хочется потрогать — ССЗБ 🙂
Еще почитать на тему можно тут, тут и тут.
Добра вам! Счастья! Здоровья! 🙂
How to Setup and Manage Log Rotation Using Logrotate in Linux
One of the most interesting (and perhaps one of the most important as well) directories in a Linux system is /var/log . According to the Filesystem Hierarchy Standard, the activity of most services running in the system are written to a file inside this directory or one of its subdirectories.
Such files are known as logs and are the key to examining how the system is operating (and how it has behaved in the past). Logs are also the first source of information where administrators and engineers look while troubleshooting.
If we look at the contents of /var/log on a CentOS/RHEL/Fedora and Debian/Ubuntu (for variety) we will see the following log files and subdirectories.
Please note that the result may be somewhat different in your case depending on the services running on your system(s) and the time they have been running.
In RHEL/CentOS and Fedora
In Debian and Ubuntu
In both cases, we can observe that some of the log names end as expected in “log”, while others are either renamed using a date (for example, maillog-20160822 on CentOS) or compressed (consider auth.log.2.gz and mysql.log.1.gz on Debian).
This is not a default behavior based on the chosen distribution but can be changed at will using directives in the configuration files, as we will see in this article.
If logs were kept forever, they would eventually end up filling the filesystem where /var/log resides. In order to prevent that, the system administrator can use a nice utility called logrotate to clean up the logs on a periodic basis.
In a few words, logrotate will rename or compress the main log when a condition is met (more about that in a minute) so that the next event is recorded on an empty file.
In addition, it will remove “old” log files and will keep the most recent ones. Of course, we get to decide what “old” means and how often we want logrotate to clean up the logs for us.
Installing Logrotate in Linux
To install logrotate, just use your package manager:
---------- On Debian and Ubuntu ---------- # aptitude update && aptitude install logrotate ---------- On CentOS, RHEL and Fedora ---------- # yum update && yum install logrotate
It is worth and well to note that the configuration file ( /etc/logrotate.conf ) may indicate that other, more specific settings may be placed on individual .conf files inside /etc/logrotate.d.
This will be the case if and only if the following line exists and is not commented out:
We will stick with this approach, as it will help us to keep things in order, and use the Debian box for the following examples.
Configure Logrotate in Linux
Being a very versatile tool, logrotate provides plenty of directives to help us configure when and how the logs will be rotated, and what should happen right afterward.
Let’s insert the following contents in /etc/logrotate.d/apache2.conf (note that most likely you will have to create that file) and examine each line to indicate its purpose:
The first line indicates that the directives inside the block apply to all logs inside /var/log/apache2:
- weekly means that the tool will attempt to rotate the logs on a weekly basis. Other possible values are daily and monthly.
- rotate 3 indicates that only 3 rotated logs should be kept. Thus, the oldest file will be removed on the fourth subsequent run.
- size=10M sets the minimum size for the rotation to take place to 10M. In other words, each log will not be rotated until it reaches 10MB.
- compress and delaycompress are used to tell that all rotated logs, with the exception of the most recent one, should be compressed.
Let’s execute a dry-run to see what logrotate would do if it was actually executed now. Use the -d option followed by the configuration file (you can actually run logrotate by omitting this option):
# logrotate -d /etc/logrotate.d/apache2.conf
The results are shown below:
Instead of compressing the logs, we could rename them after the date when they were rotated. To do that, we will use the dateext directive. If our date format is other than the default yyyymmdd, we can specify it using dateformat.
Note that we can even prevent the rotation from happening if the log is empty with notifempty. In addition, let’s tell logrotate to mail the rotated log to the system administrator ([email protected] in this case) for his / her reference (this will require a mail server to be set up, which is out of the scope of this article).
If you want to get emails about logrotate, you can setup Postfix mail server as shown here: Install Postfix Mail Server
This time we will use /etc/logrotate.d/squid.conf to only rotate /var/log/squid/access.log:
As we can see in the image below, this log did not need to be rotated. However, when the size condition is met (size=1M), the rotated log will be renamed access.log-25082020 (if the log was rotated on August 25, 2020) and the main log (access.log) will be re-created with access permissions set to 0644 and with root as owner and group owner.
Finally, when the number of logs finally reaches 6, the oldest log will be mailed to [email protected].
Now let’s suppose you want to run a custom command when the rotation takes place. To do that, place the line with such command between the postrotate and endscript directives.
For example, let’s suppose we want to send an email to root when any of the logs inside /var/log/myservice gets rotated. Let’s add the lines in red to /etc/logrotate.d/squid.conf:
/var/log/myservice/* < monthly create 0644 root root rotate 5 size=1M postrotate echo "A rotation just took place." | mail root endscript >
Last, but not least, it is important to note that options present in /etc/logrotate.d/*.conf override those in the main configuration file in case of conflicts.
Logrotate and Cron
By default, the installation of logrotate creates a crontab file inside /etc/cron.daily named logrotate. As it is the case with the other crontab files inside this directory, it will be executed daily starting at 6:25 am if anacron is not installed.
Otherwise, the execution will begin around 7:35 am. To verify, watch for the line containing cron.daily in either /etc/crontab or /etc/anacrontab.
Summary
In a system that generates several logs, the administration of such files can be greatly simplified using logrotate. As we have explained in this article, it will automatically rotate, compress, remove, and mail logs on a periodic basis or when the file reaches a given size.
Just make sure it is set to run as a cron job and logrotate will make things much easier for you. For more details, refer to the man page.
Do you have any questions or suggestions about this article? Feel free to let us know using the comment form below.