Check if file opened bash [closed]
Hey I’m trying to write a bash script and have a question. What is the best way to check if a specific file is not opened and done being written. I need to do this inside an if statement? Pseudocode:
If abc.txt not opened Do this Else Do this Even if bash offered a way to do this, the information would be obsolete the moment the check is completed, as another process can open or close a file at any time.
In general, if the check returns false, meaning the file is not open, another process could, at that very moment, open the file and append more data. Thus, the result of the check, indicating that the file is not open, would no longer be correct. The information would be obsolete.
If your process is the only one touching the file, then you would know whether or not the file is opened by your process from information inside your process. Do you mean inside your bash script, which executes some command which writes to the file?
The standard way to fix this is to have the writer write to a temporary file first, and when it has finished, rename the temporary file to the «real» file in one atomic step. The real file only appears after all the data that’s going to be written to has been written.
But it would be better to work with the cient to implement the method @chepner described. It’s failsafe because if the writing process crashes, it won’t do the rename at the end, and you’ll then ignore the file it was writing.
2 Answers 2
With lsof command you can get the files that are currently open.
If that is not 0, the file is open. If you know the pid of the process that should be using that file, you could use it to filter the results using -p option.
The sample code shown falsely reports abc.txt is open when it is not but qabc3.txt is open, for example. Also, there is no need for wc ; grep -q will silence output and report results in its exit status.
lsof abc.txt should be sufficient to list the processes that have opened abc.txt , instead of grepping the list of all open files.
The best way to check if a file is done being written is have the writer signal that it is done. This signal can take many forms, but a common one is to only create the expected file name once all writing is complete. This is accomplished by writing to a temporary file, and only renaming it after the write as complete successfully:
cp foo bar.tmp && mv bar.tmp bar some_long_process > bar.tmp && mv bar.tmp bar Now you, as the consumer, can be assured that if bar exists at all, it is complete and ready to be used.
# Polling only used as an example; operating-system-specific # solutions that block until notification of the file's creation # are vastly preferred while [ ! -e bar ]; do echo "Bar doesn't exist yet. " sleep 1 done echo "Bar exists!" do_something_with bar How to check if a file is opened in Linux?
The thing is, I want to track if a user tries to open a file on a shared account. I’m looking for any record/technique that helps me know if the concerned file is opened, at run time. I want to create a script which monitors if the file is open, and if it is, I want it to send an alert to a particular email address. The file I’m thinking of is a regular file. I tried using lsof | grep filename for checking if a file is open in gedit, but the command doesn’t return anything. Actually, I’m trying this for a pet project, and thus the question.
4 Answers 4
The command lsof -t filename shows the IDs of all processes that have the particular file opened. lsof -t filename | wc -w gives you the number of processes currently accessing the file.
I would recommend wc -l instead of wc -w which works in more cases and lsof is not going to put more than one filename per line.
The fact that a file has been read into an editor like gedit does not mean that the file is still open. The editor most likely opens the file, reads its contents and then closes the file. After you have edited the file you have the choice to overwrite the existing file or save as another file.
You could (in addition of other answers) use the Linux-specific inotify(7) facilities.
I am understanding that you want to track one (or a few) particular given file, with a fixed file path (actually a given i-node). E.g. you would want to track when /var/run/foobar is accessed or modified, and do something when that happens
In particular, you might want to install and use incrond(8) and configure it thru incrontab(5)
If you want to run a script when some given file (on a native local, e.g. Ext4, BTRS, . but not NFS file system) is accessed or modified, use inotify incrond is exactly done for that purpose.
PS. AFAIK, inotify don’t work well for remote network files, e.g. NFS filesystems (in particular when another NFS client machine is modifying a file).
If the files you are fond of are somehow source files, you might be interested by revision control systems (like git) or builder systems (like GNU make); in a certain way these tools are related to file modification.
You could also have the particular file system sits in some FUSE filesystem, and write your own FUSE daemon.
If you can restrict and modify the programs accessing the file, you might want to use advisory locking, e.g. flock(2), lockf(3).
Perhaps the data sitting in the file should be in some database (e.g. sqlite or a real DBMS like PostGreSQL ou MongoDB). ACID properties are important .
Notice that the filesystem and the mount options may matter a lot.
You might want to use the stat(1) command.
It is difficult to help more without understanding the real use case and the motivation. You should avoid some XY problem
Probably, the workflow is wrong (having a shared file between several users able to write it), and you should approach the overall issue in some other way. For a pet project I would at least recommend using some advisory lock, and access & modify the information only thru your own programs (perhaps setuid) using flock (this excludes ordinary editors like gedit or commands like cat . ). However, your implicit use case seems to be well suited for a DBMS approach (a database does not have to contain a lot of data, it might be tiny), or some index locked file like GDBM library is handling.
Remember that on POSIX systems and Linux, several processes can access (and even modify) the same file simultaneously (unless you use some locking or synchronization).
Reading the Advanced Linux Programming book (freely available) would give you a broader picture (but it does not mention inotify which appeared aften the book was written).
How to determine, whether a file is open?
My code needs to go through files in a directory, picking only those, which are currently opened (for writing) by any other process on the system. The ideal solution would apply for all Unixes, but I’ll settle for a Linux-only. The program is written in Python, but I can add a custom C-function, if I have to — I just need to know, what API is available for this. One suggestion I found was to go through all file-descriptors under Linux /proc , resolving their links to see, if they point at the file of interest. But that seems rather heavy. I know, for example, that opening a file increases its reference count — filesystem will not deallocate blocks of an opened file even if it is deleted — until it is closed — the feature relied upon by tmpfile(3) . Perhaps, a user process can get access to these records in the kernel?
Yeah, lsof — and fuser — scan /proc . But that yields more information than I need — I don’t care, which processes have the file open. I just want to know, whether any such exist. Perhaps, this information can be obtained more cheaply, than /proc rescan?
The advantage of scanning /proc is that it is backed by direct kernel calls, not a physical file system. That gives /proc a huge performance advantage over opening and reading a directory, even just to find the names.
The advantage of scanning /proc is that it is the only way to get the information without modifying the kernel.
Is there a faster way to check if a file is in use?
I’m looking for a command line function or c function that will let me know if a file is open/in use by something. lsof and fuser do tell this, but they provide a lot of other info which results in taking up to 300ms in some situations (like when i use this code on MAC OS X, I’m devving for Linux and OS X) (I have a windows solution that takes 5ms so I’m trying to find something in Unix that also is very quick, and just returns true or false if file is in use)
2 Answers 2
If you are using this as a lock, it will not work as neither lsof or fuser prevent race conditions.
The basic process that lsof does is trawl through all processes /proc/*/fs looking for open file descriptors. This is going to take time no matter what you do.
You can do this yourself, but it is not likely to be any faster as you have to check for every open process on the system.
If what you are doing is time critical, figure out another way to do it.
- If you control the file through a program that you wrote; use a lock file.
- If you are running some command that operates on the file, look and see what documentation that command/program offers and see if it can’t make a lockfile. Failing that, see if it can’t make a file with its PID inside it. Then you can look at /proc//fs to see if your file is currently open or not. Looking at only one processes open file descriptors will be much faster then mapping across all of them.
- Otherwise in order to help you I am going to need more information about what you are doing.
You gave more information in a comment that you want to determine if Firefox is running on a given system. The best way to do this is to look for Firefox’s lock files. These are stored in default locations specified on the Mozilla wiki.
For example, on linux, have your program do the following:
- open up the ~/.mozilla/firefox/ directory.
- List all directories, filtering for directories ending in .default . (I think all profiles end with .default , if not just crawl into every directory.)
- In each directory above, look for the existence of a file named lock or .parentlock . If you see one or both files, Firefox is open.
This algorithm ought to execute faster than what you do on windows currently.