make a log file
Is there any way to make a log file for maintaining some data in /var/log/ with the help of some library function or system call in c language in linux. And I also want to know the standards that we should follow to write and process log. Thanks
Please note that programming questions are generally off-topic here; you should ask them on Stack Overflow. An explanation of the API is ok here, but explanations of how to use it in C or other programming language normally belongs on Stack Overflow.
4 Answers 4
The standard way to log from a C program is syslog .
Start by including the header file:
Then early in your program, you should configure syslog by calling openlog :
openlog("programname", 0, LOG_USER); The first argument is the identification or the tag, which is automatically added at the start of each message. Put your program’s name here.
The second argument is the options you want to use, or 0 for the normal behavior. The full list of options is in man 3 syslog . One you might find useful is LOG_PID , which makes syslog also record the process id in the log message.
Then, each time you want to write a log message, you call syslog :
The first argument is the priority. The priority ranges from DEBUG (least important) to EMERG (only for emergencies) with DEBUG , INFO , and ERR being the most commonly used. See man 3 syslog for your options.
The second and third arguments are a format and a message, just like printf.
Which log file this appears in depends on your syslog settings.
With a default setup, it probably goes into /var/log/messages .
You can set up a custom log file by using one of the facilities in the range LOG_LOCAL0 to LOG_LOCAL7 .
openlog("programname", 0, LOG_USER); openlog("programname", 0, LOG_LOCAL0); openlog("programname", 0, LOG_LOCAL1); and adding a corresponding entry to /etc/syslog.conf , e.g.
local1.info /var/log/programname.log and restarting the syslog server, e.g.
The .info part of local1.info above means that all messages that are INFO or more important will be logged, including INFO , NOTICE , ERR (error), CRIT (critical), etc., but not DEBUG .
Or, if you have rsyslog , you could try a property-based filter, e.g.
:syslogtag, isequal, "programname:" /var/log/programname.log The syslogtag should contain a «:».
Or, if you are planning on distributing your software to other people, it’s probably not a good idea to rely on using LOG_LOCAL or an rsyslog filter.
In that case, you should use LOG_USER (if it’s a normal program) or LOG_DAEMON (if it’s a server), write your startup messages and error messages using syslog , but write all of your log messages to a file outside of syslog . For example, Apache HTTPd logs to /var/log/apache2/* or /var/log/httpd/* , I assume using regular open / fopen and write / printf calls.
Write to custom log file from a Bash script
In Linux, I know how to write a simply message to the /var/log/messages file, in a simple shell script I created:
I want to stop throwing messages into the default /var/log/messages file, and create my own. I tried this:
#!/bin/bash logger "have more fun" > /var/log/mycustomlog It still logs to /var/log/messages . It did create the /var/log/mycustomlog , but it’s empty. Anyone see what I’m missing?
Redirecting standard output doesn’t work because logger doesn’t write to standard output; it writes to the file configured by syslog(3) to receive log messages.
5 Answers 5
logger logs to syslog facilities. If you want the message to go to a particular file you have to modify the syslog configuration accordingly. You could add a line like this:
and restart syslog. Then you can log like this:
logger -p local7.info "information message" logger -p local7.err "error message" and the messages will appear in the desired logfile with the correct log level.
Without making changes to the syslog configuration you could use logger like this:
logger -s "foo bar" >> /var/log/mycustomlog That would instruct logger to print the message to STDERR as well (in addition to logging it to syslog), so you could redirect STDERR to a file. However, it would be utterly pointless, because the message is already logged via syslog anyway (with the default priority user.notice ).
For anyone else who was confused at the use of the number 2 after the string in the last example, 1 redirects stdout, and 2 redirects stderr. More info available here: tldp.org/LDP/abs/html/io-redirection.html
And for anyone who doesn’t know how to edit the syslog configuration, here it is on Ubuntu: askubuntu.com/questions/42152/where-is-syslog-conf
What does the — before the file path indicate? I’ve tried googling around but can’t find anything about it. Is it required?
@chepner make a good point that logger is dedicated to logging messages.
I do need to mention that @Thomas Haratyk simply inquired why I didn’t simply use echo .
At the time, I didn’t know about echo, as I’m learning shell-scripting , but he was right.
My simple solution is now this:
#!/bin/bash echo "This logs to where I want, but using echo" > /var/log/mycustomlog The example above will overwrite the file after the >
So, I can append to that file with this:
#!/bin/bash echo "I will just append to my custom log file" >> /var/log/customlog - on a side note, it’s simply my personal preference to keep my personal logs in /var/log/ , but I’m sure there are other good ideas out there. And since I didn’t create a daemon, /var/log/ probably isn’t the best place for my custom log file. (just saying)
I agree that this is a good approach but the log file will keep growing and there is no mechanism of archiving or deleting old logs. If you log frequently, then this can take considerable amount of storage.
There’s good amount of detail on logging for shell scripts via global varaibles of shell. We can emulate the similar kind of logging in shell script: http://www.cubicrace.com/2016/03/efficient-logging-mechnism-in-shell.html The post has details on introdducing log levels like INFO , DEBUG, ERROR. Tracing details like script entry, script exit, function entry, function exit.
If you see the man page of logger:
LOGGER(1) BSD General Commands Manual LOGGER(1)
NAME logger — a shell command interface to the syslog(3) system log module
SYNOPSIS logger [-isd] [-f file] [-p pri] [-t tag] [-u socket] [message . ]
DESCRIPTION Logger makes entries in the system log. It provides a shell command interface to the syslog(3) system log module.
It Clearly says that it will log to system log. If you want to log to file, you can use «>>» to redirect to log file.
This is an incorrect interpretation of the man page. logger uses syslog or (in the case of most Linux systems) rsyslog which can be configured to place log messages where ever you like.
I did it by using a filter. Most linux systems use rsyslog these days. The config files are located at /etc/rsyslog.conf and /etc/rsyslog.d .
Whenever I run the command logger -t SRI some message , I want «some message» to only show up in /var/log/sri.log .
To do this I added the file /etc/rsyslog.d/00-sri.conf with the following content.
# Filter all messages whose tag starts with SRI # Note that 'isequal, "SRI:"' or 'isequal "SRI"' will not work. # :syslogtag, startswith, "SRI" /var/log/sri.log # The stop command prevents this message from getting processed any further. # Thus the message will not show up in /var/log/messages. # & stop Then restart the rsyslogd service:
systemctl restart rsyslog.service Here is a shell session showing the results:
[root@rpm-server html]# logger -t SRI Hello World! [root@rpm-server html]# cat /var/log/sri.log Jun 5 10:33:01 rpm-server SRI[11785]: Hello World! [root@rpm-server html]# [root@rpm-server html]# # see that nothing shows up in /var/log/messages [root@rpm-server html]# tail -10 /var/log/messages | grep SRI [root@rpm-server html]# Write to and manage syslog messages with logger and rsyslog
Syslog is the target where you want all log message to go on all systems that you manage. Almost all Linux distributions use a syslog implementation to gather messages. Recently, rsyslog became the most used syslog-implementation for Linux. Messages can be saved locally or sent to a remote syslog server. When creating your own applications or tools or when you want to log messages coming from processes that don’t support writing to syslog directly, you can use Logger.
Logger is a small and easy to use tool. It allows you to forward messages that come on stdout or stderr to syslog and that’s it. The messages can be tagged and/or contain the pid of the application writing the message. By only forwarding the messages to syslog, they will probably end up in /var/log/messages which could be fine for what you want to do. I’ll also explain how to define rules for rsyslog to move the message to a separate file.
On CentOS and Debian, logger, rsyslog and logrotate are a standard part of the minimal install so in most cases those packages don’t need to be installed. The commands used in this post should be completely interchangeable between most distributions.
Basic usage of logger
Using logger is quite simple. In the most simple form, you can write a message to syslog as follows:
[jensd@node01 ~]$ logger testmessage [jensd@node01 ~]$ sudo tail -1 /var/log/messages Dec 2 21:19:09 node01 jensd: testmessage
When we want to provide a little more information:
[jensd@node01 ~]$ logger -i -t testid testmessage [jensd@node01 ~]$ sudo tail -1 /var/log/messages Dec 2 21:19:55 node01 testid[2401]: testmessage
Putting the messages in a separate logfile
As you can see in the previous examples, the message which we sent to syslog went to file /var/log/messages. On most distributions, uncategorized messages end up in /var/log/messages or /var/log/syslog. The target where a message is written to is configured in /etc/rsyslog.conf. It is possible to send specific messages to a specific file or even a remote syslog-server.
To send messages that have the testid identifier to a file /var/log/testlog, we need to add the following to /etc/rsyslog.conf
#### RULES #### $template testlog, "/var/log/testlog" if $programname contains 'testid' then ?testlog & ~
- Line 1: the starting point of the specific rules in the configuration file, it’s there by default on most distributions.
- Line 2: definition of a template for the file, it can also contain variables like %HOSTNAME% or %programname%
- Line 3: the condition for a message to be sent to the file define in the template
- Line 4: prevents messages that matched the condition on line 3 to be processed further. Without this line, the message will end up in /var/log/testlog and in /var/log/messages.
After changing the configuration file /etc/rsyslog.conf, we need to restart rsyslog for the changes to take effect:
[jensd@node01 ~]$ sudo vi /etc/rsyslog.conf [jensd@node01 ~]$ sudo systemctl restart rsyslog [jensd@node01 ~]$ logger -i -t testid testmessage [jensd@node01 ~]$ tail /var/log/testlog Dec 2 21:28:14 cen testid[2453]: testmessage [jensd@node01 ~]$ ls -l /var/log/testlog -rw-r--r--. 1 root root 46 Dec 2 21:28 /var/log/testlog
As you can see, after resending our testmessage, the message ended up in /var/log/testlog as we defined in the configuration file of rsyslog.
As you can also see in the above output, the file got created and is owned by root which can be annoying when other processes also need to write to the logfile or it needs to be cleared by another process, not running as root. To let the file be created and owned by another user or group, we can change the configuration in /etc/rsyslog.conf to include the following:
#### RULES #### $FileOwner jensd $FileCreateMode 0660 $FileGroup jensd $template testlog, "/var/log/testlog" if $programname contains 'testid' then ?testlog & ~ $FileOwner root $FileCreateMode 0644 $FileGroup root
After the rules specific for our logfile, it’s important to set the owner/group and permissions back to their defaults to not break other log-operations.
After restarting rsyslog, you can see that the owner changed as we requested.
[jensd@node01 ~]$ sudo vi /etc/rsyslog.conf [jensd@node01 ~]$ sudo rm /var/log/testlog [jensd@node01 ~]$ sudo systemctl restart rsyslog [jensd@node01 ~]$ logger -i -t testid testmessage [jensd@node01 ~]$ tail /var/log/testlog Dec 2 21:37:53 cen testid[2482]: testmessage [jensd@node01 ~]$ ls -l /var/log/testlog -rw-r-----. 1 jensd jensd 46 Dec 2 21:37 /var/log/testlog
Logger also accepts stdin as a source of the message to write to the log, so it’s a small but powerful tool to log messages to syslog from processes that do not support that functionality directly.
[jensd@node01 ~]$ echo "message2"|logger -i -t testid [jensd@node01 ~]$ tail /var/log/testlog Dec 2 21:37:53 cen testid[2482]: testmessage Dec 2 21:43:26 cen testid[2487]: message2
This time, I managed to write a small post, hopefully it can help you in unifying and bundling log messages for any possible script or process.