Counting open files per process
I’m working on an application that monitors the processes’ resources and gives a periodic report in Linux, but I faced a problem in extracting the open files count per process. This takes quite a while if I take all of the files and group them according to their PID and count them. How can I take the open files count for each process in Linux?
5 Answers 5
Have a look at the /proc/ file system:
To do this for all processes, use this:
cd /proc for pid in 6* do echo "PID = $pid with $(ls /proc/$pid/fd/ | wc -l) file descriptors" done As a one-liner (filter by appending | grep -v «0 FDs» ):
for pid in /proc/3*; do printf "PID %6d has %4d FDs\n" $(basename $pid) $(ls $pid/fd | wc -l); done As a one-liner including the command name, sorted by file descriptor count in descending order (limit the results by appending | head -10 ):
for pid in /proc/9*; do p=$(basename $pid); printf "%4d FDs for PID %6d; command=%s\n" $(ls $pid/fd | wc -l) $p "$(ps -p $p -o comm=)"; done | sort -nr Credit to @Boban for this addendum:
You can pipe the output of the script above into the following script to see the ten processes (and their names) which have the most file descriptors open:
. done | sort -rn -k5 | head | while read -r _ _ pid _ fdcount _ do command=$(ps -o cmd -p "$pid" -hc) printf "pid = %5d with %4d fds: %s\n" "$pid" "$fdcount" "$command" done Here’s another approach to list the top-ten processes with the most open fds, probably less readable, so I don’t put it in front:
find /proc -maxdepth 1 -type d -name '8*' \ -exec bash -c "ls <>/fd/ | wc -l | tr '\n' ' '" \; \ -printf "fds (PID = %P), command: " \ -exec bash -c "tr '\0' ' ' < <>/cmdline" \; \ -exec echo \; | sort -rn | head Of course, you will need to have root permissions to do that for many of the processes. Their file descriptors are kind of private, you know 😉
/proc/$pid/fd lists descriptor files, that is slightly different of «open files» as we can have memory map and other unusual file objects.
This extends the answer and turns pids to command names: for pid in 4*; do echo «PID = $pid with $(ls /proc/$pid/fd/ 2>/dev/null | wc -l) file descriptors»; done | sort -rn -k5 | head | while read -r line; do pid= echo $line | awk ‘
Yeah, well. Instead of parsing the original output and then call ps again for each process to find out its command, it might make more sense to use /proc/$pid/cmdline in the first loop. While technically it is still possible for a process to disappear between the evaluating of 2* and the scanning of its disc, this is less likely.
Executing command=$(ps -o cmd -p «$pid» -hc) gave me Warning: bad syntax, perhaps a bogus ‘-‘ . It worked running as command=$(ps -o cmd -p «$pid» hc) .
ps aux | sed 1d | awk '' | xargs -I <> bash -c <> For Fedora, it gives: lsof: WARNING: can’t stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs Output information may be incomplete. lsof: no pwd entry for UID 65535
I used this to find top filehandler-consuming processes for a given user (username) where dont have lsof or root access:
for pid in `ps -o pid -u username` ; do echo "$(ls /proc/$pid/fd/ 2>/dev/null | wc -l ) for PID: $pid" ; done | sort -n | tail How can I take the open files count for each process in Linux?
if you’re running it from root (e.g. prefixing the command with sudo -E env PATH=$PATH ), otherwise it’ll only return file descriptor counts per process whose /proc//fd you may list. This will give you a big JSON document/tree whose nodes look something like:
The content of fd dictionary is counts per file descriptor type. The most interesting ones are probably these (see procfile.Fd description or man fstat for more details):
I’m the author of Procpath, which is a tool that provides a nicer interface to procfs for process analysis. You can record a process tree’s procfs stats (in a SQLite database) and plot any of them later. For instance this is how my Firefox’s process tree (root PID 2468) looks like with regards to open file descriptor count (sum of all types):
procpath --logging-level ERROR record -f stat,fd -i 1 -d ff_fd.sqlite \ '$..children[?(@.stat.pid == 2468)]' # Ctrl+C procpath plot -q fd -d ff_fd.sqlite -f ff_df.svg 
If I’m interested in only a particular type of open file descriptors (say, sockets) I can plot it like this:
procpath plot --custom-value-expr fd_sock -d ff_fd.sqlite -f ff_df.svg how to find the number of open files per process
we have kafka service ( as systemctl service ) and we configured in that service number of open files example:
[Service] LimitMEMLOCK=infinity LimitNOFILE=1500000 Type=forking User=root Group=kafka now when service is up , we want to understand the consuming of number of files by kafka services from googled , I understand from — https://www.cyberciti.biz/faq/howto-linux-get-list-of-open-files/ that we can use the command fstat in order to capture the number of open files as
since we are using production RHEL 7.6 secured server , then its not clear if fstat can be installed on our server therefore we want to learn about other ideas? appreciate to get other approach other approach as suggest is by — ls «/proc/$pid/fd» but here is real example from my machine
ls /proc/176909/fd |more 0 1 10 100 1000 10000 10001 10002 10003 10004 10005 10006 10007 10008 10009 1001 10010 10011 10012 . . . The link you gave mentions fstat for FreeBSD and ls «/proc/$pid/fd» for Linux. RHEL is a Linux distribution, not FreeBSD.
Run ls -l . You will see that those are symlinks to various files, devices, sockets, etc. You could do ls -1 /proc/176909/fd | wc -l . ls -1 /proc/176909/fd puts every listed item out on a separate line in one column. wc -l counts the number of lines. So this would give you the total number of items in that directory. I think you should just keep reading the link you posted as it says all of this.
2 Answers 2
As mentioned in comments you can use wc command:
-l count the number of lines (result of ls command)
The LimitNOFILE directive in systemd (see man systemd.exec ) corresponds to the RLIMIT_NOFILE resource limit as set with setrlimit() (see man setrlimit ). Can be set in some shells with ulimit -n or limit descriptors .
This specifies a value one greater than the maximum file descriptor number that can be opened by this process. Attempts (open(2), pipe(2), dup(2), etc.) to exceed this limit yield the error EMFILE. (Historically, this limit was named RLIMIT_OFILE on BSD.)
So it’s not strictly speaking the limit of number of open file descriptors (let alone open files) in that a process with that limit could have more open files if it had fds above the limit prior to that limit being set (or inherited upon creation ( clone() / fork() )) and could not get a fd above the limit even if it had very few opened fds.
On Linux, /proc//fd is a special directory that contains one magic symlink file for each fd that the process has opened.
You can get their number by counting them:
in zsh for instance (or ls «/proc/$pid/fd» | wc -l as already shown by Romeo).
You can get the highest pid value by sorting them numerically in reverse and get the first.
Or with GNU ls : ls -rv «/proc/$pid/fd» | head -n1
To get a report of number of open fds for all processes, you could do something like:
(for p (/proc/) () $p/fd/*(NoN)) | sort -n More portably, you could resort to lsof :
lsof -ad0-2147483647 -Ff -p "$pid" | grep -c '^f' For the number of open file descriptors and:
lsof -ad0-2147483647 -Ff -p "$pid" | sed -n '$s/^f//p' How to display open file descriptors but not using lsof command
While this command displays the number of FD’s, how do you display the list of open file descriptors that the command above just counted?
You probably want to know if your ulimit is exceeded, right? I blogged about this under linuxintro.org/wiki/Is_my_ulimit_exceeded; most importantly, the ulimit is a per-process restriction that you can find under /proc/PID/limits and instead of lsof I would use ls /proc/PID/fd to list the process’ file descriptors.
2 Answers 2
There are two reasons lsof | wc -l doesn’t count file descriptors. One is that it lists things that aren’t open files, such as loaded dynamically linked libraries and current working directories; you need to filter them out. Another is that lsof takes some time to run, so can miss files that are opened or closed while it’s running; therefore the number of listed open files is approximate. Looking at /proc/sys/fs/file-nr gives you an exact value at a particular point in time.
cat /proc/sys/fs/file-nr is only useful when you need the exact figure, mainly to check for resource exhaustion. If you want to list the open files, you need to call lsof , or use some equivalent method such as trawling /proc/*/fd manually.
Hi thanks for giving a good explanation Gilles. I tried ls /proc/*/fd and got all the open fd’s at that time. Its producing an output with some color coding, I’ll just have to look at the manual.
@dimas /proc/*/fd directories contain symbolic links to the open files. For visual inspection, use ls -l . For automated treatment, use readlink to extract the link target.
Just use ls -l but i’ll experiment with readlink. I tried other /proc/PID/maps and other options as specified here kernel.org/doc/man-pages/online/pages/man5/proc.5.html. Thanks again for the additional info.
/proc/sys/fs/file-nr gives me 3872 (and two other numbers). How can this be the count of files I have open if ulimit -n shows me 1024?
@ThorstenStaerk All settings of setrlimit (the system call underlying the ulimit shell command) are per-process. They affect only the process that makes the call (and indirectly the processes that it later forks).
Process information is kept dynamically by the system in directories under /proc. For example the process with PID 1234 will have a directory called /proc/1234.
There are quite a bit of information in there but right now you are interested in the /proc/1234/fd subdirectory.
NOTE: You need to have root permissions to view or open files for processes that you do not own, as well as for SetUID processes.
root@johan-HP-ProBook-6560b-LG654EA-ACQ:/proc# ls -l 2443/fd total 0 lr-x------ 1 johan johan 64 Feb 27 10:26 0 -> pipe:[13637] l-wx------ 1 johan johan 64 Feb 27 10:26 1 -> /home/johan/.xsession-errors lrwx------ 1 johan johan 64 Feb 27 10:26 10 -> anon_inode:[eventfd] lrwx------ 1 johan johan 64 Feb 27 10:26 11 -> anon_inode:[eventfd] lrwx------ 1 johan johan 64 Feb 27 10:26 12 -> socket:[39495] lrwx------ 1 johan johan 64 Feb 27 10:26 13 -> anon_inode:[eventfd] lr-x------ 1 johan johan 64 Feb 27 10:26 14 -> anon_inode:inotify lrwx------ 1 johan johan 64 Feb 27 10:26 15 -> anon_inode:[eventfd] l-wx------ 1 johan johan 64 Feb 27 10:26 16 -> pipe:[37885] lr-x------ 1 johan johan 64 Feb 27 10:26 17 -> pipe:[37886] l-wx------ 1 johan johan 64 Feb 27 10:26 2 -> /home/johan/.xsession-errors l-wx------ 1 johan johan 64 Feb 27 10:26 21 -> pipe:[167984] lr-x------ 1 johan johan 64 Feb 27 10:26 22 -> pipe:[167985] l-wx------ 1 johan johan 64 Feb 27 10:26 23 -> pipe:[170009] lr-x------ 1 johan johan 64 Feb 27 10:26 24 -> pipe:[170010] lrwx------ 1 johan johan 64 Feb 27 10:26 3 -> anon_inode:[eventfd] lr-x------ 1 johan johan 64 Feb 27 10:26 4 -> pipe:[14726] lrwx------ 1 johan johan 64 Feb 27 10:26 5 -> socket:[14721] l-wx------ 1 johan johan 64 Feb 27 10:26 6 -> pipe:[14726] lrwx------ 1 johan johan 64 Feb 27 10:26 7 -> socket:[14730] lrwx------ 1 johan johan 64 Feb 27 10:26 8 -> socket:[13984] lrwx------ 1 johan johan 64 Feb 27 10:26 9 -> socket:[14767] root@johan-HP:/proc# cat 2443/fdinfo/2 pos: 1244446 flags: 0102001 Also have a look at the rest of the files under /proc . a lot of useful information from the system resides here.