Linux file descriptors limit

Upper limit of file descriptor in Linux

Well, in that case, you are good with using ulimit for setting per process fd limits and as in my answer, you can use file-max or file-nr(based on which one suits you better) for system-wide fd limits. You might also want to increase the /proc/sys/kernel/pid_max limit if you are going to run several processes.

2 Answers 2

You want to look at /proc/sys/fs/file-max instead.

From the recent linux/Documentation/sysctl/fs.txt:

file-max and file-nr:

The kernel allocates file handles dynamically, but as yet it doesn’t free them again.

The value in file-max denotes the maximum number of file- handles that the Linux kernel will allocate. When you get lots of error messages about running out of file handles, you might want to increase this limit.

Historically, the three values in file-nr denoted the number of allocated file handles, the number of allocated but unused file handles, and the maximum number of file handles. Linux 2.6 always reports 0 as the number of free file handles — this is not an error, it just means that the number of allocated file handles exactly matches the number of used file handles.

Attempts to allocate more file descriptors than file-max are reported with printk, look for «VFS: file-max limit reached».

The 2.6 kernel uses a rule of thumb to set file-max based on the amount of memory in the system. A snippet from fs/file_table.c in the 2.6 kernel:

/* * One file with associated inode and dcache is very roughly 1K. * Per default don't use more than 10% of our memory for files. */ n = (mempages * (PAGE_SIZE / 1024)) / 10; files_stat.max_files = max_t(unsigned long, n, NR_FILE); 

The files_stat.max_files is the setting of fs.file-max . This ends up being about 100 for every 1MB of ram.(10%)

Источник

Fixing the “Too many open files” Error in Linux

The Kubernetes ecosystem is huge and quite complex, so it’s easy to forget about costs when trying out all of the exciting tools.

To avoid overspending on your Kubernetes cluster, definitely have a look at the free K8s cost monitoring tool from the automation platform CAST AI. You can view your costs in real time, allocate them, calculate burn rates for projects, spot anomalies or spikes, and get insightful reports you can share with your team.

Connect your cluster and start monitoring your K8s costs right away:

1. Overview

When working with Linux servers, we may encounter the “Too many open files” error. In this article, we’ll go over what this error means and how we can fix it.

2. What Is a File Descriptor?

From regular data to network sockets, everything is a file in Linux! A file descriptor is a non-negative integer identifier for an open file in Linux. Each process has a table of open file descriptors where a new entry is appended upon opening a new file.

For example, what happens when we cat a file? It opens the file passed as an argument via the open() system call and gets assigned a file descriptor for it. Then, it interacts with the file through the file descriptor – in this case, just for showing its contents – and finally, closes it via the close() system call.

A process has three file descriptors open by default, denoted by 0 for stdin, 1 for stdout, and 2 for stderr.

3. Checking Open File Descriptors

We can check the number of file descriptors used by various agents to determine where our resources are being used.

3.1. Global Usage

If we want to check the total number of file descriptors open on the system, we can use an awk one-liner to find this in the first field of the /proc/sys/fs/file-nr file:

$ awk '' /proc/sys/fs/file-nr 2944

3.2. Per-Process Usage

We can use the lsof command to check the file descriptor usage of a process. Let’s take the caddy web server as an example:

$ sudo lsof -p $(pidof caddy) COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME caddy 135 caddy cwd DIR 254,1 4096 1023029 /etc/sv/caddy caddy 135 caddy rtd DIR 254,1 4096 2 / caddy 135 caddy txt REG 254,1 33595392 1542819 /usr/bin/caddy caddy 135 caddy 0u CHR 5,1 0t0 12 /dev/console caddy 135 caddy 1u CHR 5,1 0t0 12 /dev/console caddy 135 caddy 2w FIFO 0,9 0t0 119 pipe caddy 135 caddy 3u a_inode 0,10 0 6 [eventpoll:2,4,6,7,8,9,10,11,13,15,16,17,18,19,20,21,22,23,24,25,27,28,29,30,31,32,33,34] caddy 135 caddy 4r FIFO 0,9 0t0 171 pipe caddy 135 caddy 5w FIFO 0,9 0t0 171 pipe caddy 135 caddy 6u IPv4 234 0t0 TCP localhost.localdomain:2019 (LISTEN) caddy 135 caddy 7u IPv6 240 0t0 TCP *:443 (LISTEN) caddy 135 caddy 8u IPv6 241 0t0 TCP *:80 (LISTEN) caddy 135 caddy 9u IPv6 3819034 0t0 TCP 66.46.134.82.baeldung.com:443->google.com:53222 (ESTABLISHED)

We can see the exact file that a given file descriptor belongs to under the NAME column, while its type is located under the TYPE column. Looking at the entries with IPv6 as their type, we can conclude that even network sockets take up file descriptors, just like regular files.

4. File Descriptor Limits

A process has certain limits with regard to the number of file descriptors it can have open at a time. One is a soft limit, which can be changed by any unprivileged user and can never exceed the hard limit. An unprivileged user can lower the hard limit but not raise it again, whereas a privileged user such as root can raise and lower it as required.

4.1. Per-Session Limit

We use the ulimit command with the -Sn flag to check the soft limit, and the -Hn flag to check the hard limit for the current session:

$ ulimit -Sn 1024 $ ulimit -Hn 4096

4.2. Per-Process Limit

We can check a process’s limits, given its PID, via the procfs filesystem:

$ pid=31540 $ grep "Max open files" /proc/$pid/limits Max open files 1024 4096 files

The second and third fields correspond to the soft and hard limits, respectively.

4.3. Global Limit

There’s a system-wide limit to the total number of file descriptors that can be open by all the processes combined. This limit is stored in the /proc/sys/fs/file-max file:

$ cat /proc/sys/fs/file-max 1634185

5. Increasing File Descriptor Limits

Now that we have a good understanding of the idea behind the “Too many open files” error, let’s go over various ways to solve it. We can verify these changes with the commands mentioned in the previous section. We use 500000 to refer to the desired limit under this section’s examples.

5.1. Temporarily (Per-Session)

Let’s try to configure the limit for our current session with the ulimit -n command:

$ ulimit -n 4096 $ ulimit -n 4096 $ ulimit -n 8192 sh: error setting limit: Operation not permitted

We cannot set the limit above 4096 as that is the hard limit in this case. And as noted above, only a privileged user can change the hard limit.

5.2. Per-User

We can change the soft and hard limits globally for all processes by appending a few lines to the /etc/security/limits.conf file and re-logging in:

* hard nofile 500000 * soft nofile 500000 root hard nofile 500000 root soft nofile 500000

In the first field, we specify the user that the limit affects. We set the limit for all users on the system with the * glob.

Additionally, we might need to append a line to /etc/pam.d/common-session on some systems:

session required pam_limits.so

5.3. Per-Service

On systemd-based distributions, we use the systemctl command to configure the limits for specific services. Taking apache as an example, let’s create a filelimit.conf file for it with tee:

$ sudo mkdir -p /etc/systemd/system/apache.service.d/ $ sudo tee /etc/systemd/system/apache.service.d/filelimit.conf  

$ sudo systemctl daemon-reload $ sudo systemctl restart apache.service