What are pid and lock files for?
I often see that programs specify pid and lock files. And I’m not quite sure what they do. For example, when compiling nginx:
--pid-path=/var/run/nginx.pid \ --lock-path=/var/lock/nginx.lock \ 3 Answers 3
pid files are written by some programs to record their process ID while they are starting. This has multiple purposes:
- It’s a signal to other processes and users of the system that that particular program is running, or at least started successfully.
- It allows one to write a script really easy to check if it’s running and issue a plain kill command if one wants to end it.
- It’s a cheap way for a program to see if a previous running instance of it did not exit successfully.
Mere presence of a pid file doesn’t guarantee that that particular process id is running, of course, so this method isn’t 100% foolproof but «good enough» in a lot of instances. Checking if a particular PID exists in the process table isn’t totally portable across UNIX-like operating systems unless you want to depend on the ps utility, which may not be desirable to call in all instances (and I believe some UNIX-like operating systems implement ps differently anyway).
The idea with lock files is the following: the purpose is for two (well-behaved) separate instances of a program, which may be running concurrently on one system, don’t access something else at the same time. The idea is before the program accesses its resource, it checks for presence of a lock file, and if the lock file exists, either error out or wait for it to go away. When it doesn’t exist, the program wanting to «acquire» the resource creates the file, and then other instances that might come across later will wait for this process to be done with it. Of course, this assumes the program «acquiring» the lock does in fact release it and doesn’t forget to delete the lock file.
This works because the filesystem under all UNIX-like operating systems enforces serialization, which means only one change to the filesystem actually happens at any given time. Sort of like locks with databases and such.
Operating systems or runtime platforms typically offer synchronization primitives and it’s usually a better decision to use those instead. There may be situations such as writing something that’s meant to run on a wide variety of operating systems past and future without a reliable library to abstract the calls (such as possibly sh or bash based scripts meant to work in a wide variety of unix flavors) — in that case this scheme may be a good compromise.
How to Lock a Text File in Linux Using flock Command
Before we explore the techniques/approaches of locking a text file under a Linux operating system environment, we should first understand the logic behind Linux’s file locking mechanism.
Linux’s file locking mechanism restricts/controls file access among multiple processes. When a text file is successfully locked, only one process can access it on a specific time schedule.
Before proceeding with this article, please understand that file locking is very much different from file encryption or file access control where a passphrase or password is needed to control user access to your files.
When a file is locked under a Linux operating system environment, a mutual exclusion event is created where only a single process can access it at a time.
Problem Statement
The Linux operating system will automatically block any write action attempts directed to an open file being written on by another process. However, what if you want to revoke the read and/or write permissions already invoked by the first process that has your file open and in write mode? Is there a workaround to this problem?
Moreover, we might also want to lock our file so that no other process interferes or attempt to disrupt the write mode status already initiated. This article will walk us through a viable solution to lock a text file in Linux.
Sample Reference File
For this tutorial to be more engaging and worthwhile, we should have some reference text files. We will concentrate on the text files inside the following directory:
For instance, let us open the file output.txt and start writing on it.
While this file is still open, let us try to open it from another command line tab.
You will get the following response:
The above screen capture relays the PID (133110) of the process of working on the text file with the option of opening the file while it is still in that process’s write mode.
By keying in Y from our keyboard, we will have opened this text file and handed it over to a different process with exclusive write mode access.
Therefore, a file modification by user 1 will lead to the following prompt on user 2 while attempting to save the file.
This instance is a perfect representation of two different users on the network working on a single file.
Lock a Text File Using Linux’s flock Utility
To solve this issue, we need the aid of Linux’s flock utility. Since the util-linux package hosts flock command, it should be available in almost all Linux OS distributions. This command manages specific file/directory locks via the Linux command line environment.
To lock a text file in Linux, we will reference the following syntax:
$ flock -x PATH_TO_FILE_TO_LOCK -c COMMAND
The -x option is for obtaining a write lock to the targeted file. Let us attempt to lock the sample output.txt text file. The -c option will enable us to pass a single Linux supported command e.g. cat command.
$ flock -x /home/dnyce/LinuxShellTips_Files/output.txt -c cat
The terminal instance above will remain active to signify that the text file has been locked.
If we open another terminal (while this terminal instance is still running) and execute another flock command on this same file, we should be denied access to it.
$ flock -w .007 /home/dnyce/LinuxShellTips_Files/output.txt -c echo; /bin/echo $? 1
The -w option is used to relay the wait time of .007 seconds before a lock is placed on the text file. We then execute echo $? to output the exit status of this command.
An exit status of 0 implies that the command was executed successfully and an exit status of 1 implies the command could not be executed due to an error. In this case, the text file is under lock by another process.
To further confirm that the text file is under lock, we can use the lslocks commands to list all active local system locks.
As you can see, our file is present.
Unlocking a Text File in Linux
Canceling the initial flock command (Ctrl+c) or closing the text file should release the lock making it possible to successfully run the following command on the secondary terminal.
$ flock -w .007 /home/dnyce/LinuxShellTips_Files/output.txt -c echo; /bin/echo $?
The exit status of 0 implies that the lock is no longer applicable to the text file.
We have learned how to lock a text file in Linux so that only one process can use it (write mode) at a time. The implementation of this article is particularly useful when different users on a network are accessing a single file.
More on flock command can be found on its man page.
How to list processes locking file?
Using flock , several processes can have a shared lock at the same time, or be waiting to acquire a write lock. How do I get a list of these processes? That is, for a given file X, ideally to find the process id of each process which either holds, or is waiting for, a lock on the file. It would be a very good start though just to get a count of the number of processes waiting for a lock.
4 Answers 4
lslocks , from the util-linux package, does exactly this.
In the MODE column, processes waiting for a lock will be marked with a * .
Apt-cache says that util-linux is already the newest version (2.20.1-1ubuntu3), but I don’t have lslocks; is there a repo I can use that will give me it?
Looks like this was added in 2.22, so Ubuntu’s version is too old. Presumably a new version will be available eventually. (This is also the case with RHEL 6 or CentOS.) You could build it yourself, or you could use the lsof approach Joel Davis suggests.
lslocks reads /proc/locks , in a pinch you can read that directly yourself, with the caveat that files are identified by device and inode rather than name. Since you know the file, that ought not be a problem. Blocked entries have a -> prefix before the lock type column (thus adding a column to that line).
in my case lslocks showed that the file was locked by a PID that did not exist. But lsof pointed out the true culprit, an instance of ssh-agent ¯\_(ツ)_/¯ it probably got in when I tried flock -c that launched a shell..
Two possibilities: lsof (my preference) or lslk (specifically for file locks):
[root@policyServer ~]# lslk | grep "master.lock" SRC PID DEV INUM SZ TY M ST WH END LEN NAME master 1650 253,0 12423 33 w 0 0 0 0 0 /var/lib/postfix/master.lock [root@policyServer ~]# lsof | grep "master.lock" master 1650 root 10uW REG 253,0 33 12423 /var/lib/postfix/master.lock Output of lslk is self-expanatory but lsof puts the lock description in the «FD» column (which is 10uW above). From the man page:
The mode character is followed by one of these lock characters, describing the type of lock applied to the file: N for a Solaris NFS lock of unknown type; r for read lock on part of the file; R for a read lock on the entire file; w for a write lock on part of the file; W for a write lock on the entire file; u for a read and write lock of any length; U for a lock of unknown type; x for an SCO OpenServer Xenix lock on part of the file; X for an SCO OpenServer Xenix lock on the entire file; space if there is no lock. So the «FD» column of lsof above breaks down to:
10 The literal descriptor of this open file. What’s linked to by /proc/1650/fd/10
u File is open for reading and writing
W program has a write lock on the file.