Number of Files Opened: Difference between revisions

From Leo's Notes
This page was last edited on 5 October 2022, at 00:22.
← Older editNewer edit →
VisualWikitext
No edit summary
No edit summary
Line 1: Line 1:
Linux allows you to control how many opened file descriptor an user can have at any one time.
Linux allows you to control how many opened file descriptors a user can have at any one time.


== System-Wide Settings ==
== System-wide settings ==


To show the number of files opened on a system, run:
To show the number of files opened on a system, run:
{{highlight|lang=terminal|code=
{| class="wikitable"
!Description
!Command
|-
|Read the current file descriptor numbers. 3 values are:
 
*Current number of opened files
* Number of free allocated file descriptors
* Maximum number of file descriptors for the whole system
|{{highlight|lang=terminal|code=
# cat /proc/sys/fs/file-nr
# cat /proc/sys/fs/file-nr
1536 0 793120
7552    0       9223372036854775807
}}
|-
|Maximum number of file descriptiors.
|{{Highlight
| code = # cat /proc/sys/fs/file-max
9223372036854775807
| lang = terminal
}}
}}
|-
|Set the number of file descriptors
|{{highlight|lang=terminal|code=
## Via /proc
# cat 99999 > /proc/sys/fs/file-nr


The fields are:
## Via sysctl
# Current number of opened files
# sysctl -w fs.file-max=99999
# Number of free allocated file descriptors
# Maximum number of file descriptors for the whole system


You may change this value by:
## Edit /etc/sysctl.conf so it applies on boot
{{highlight|lang=terminal|code=
# echo "fs.file-max = 99999" >> /etc/sysctl.conf
# cat <new value> > /proc/sys/fs/file-nr
## Edit {{code|/etc/sysctl.conf}} so that there is a line
## fs.file-max = <new value>
}}
}}
|}


== Per-User Settings ==
== User session setting ==
On typical Linux distributions using PAM, user's session's limits are controlled by editing {{code|/etc/security/limits.conf}} and are applied by the <code>pam_limits.so</code> module (Enabled when using <code>session required pam_limits.so</code> in the system-auth config)


You can see the limits set by running {{code|ulimit}} on the bash shell.
A user can check their current session's limits with {{code|ulimit}} on the bash shell.
{{highlight
{| class="wikitable"
!Description
!What
|-
|Max number of files opened
|{{highlight
| lang = terminal
| lang = terminal
| code = # ulimit -n
| code = # ulimit -n
1024
1024
}}
This tells you that the maximum number of files you can have in this session is 1024 files.


For normal shell login sessions, you can specify these limits in {{code|/etc/security/limits.conf}}. To change a user's soft and hard file limits, add the following lines:
## Soft limit
# ulimit -Sn
1024
 
## Hard limit
# ulimit -Hn
1024
}}This tells you that the maximum number of files you can have in this session is 1024 files.
|}
To change a user's soft and hard file limits, add the following lines:


{{Highlight
{{Highlight
Line 38: Line 68:
username hard nofile 32768
username hard nofile 32768
| lang = text
| lang = text
}}
}}The user will have to re-login for these new limits to be applied.


== Systemd service ==
== Systemd service ==
Line 61: Line 91:
[Install]
[Install]
WantedBy=multi-user.target
WantedBy=multi-user.target
| lang = text
}}
=== Checking if limits are set properly ===
Each process's limits are available under <code>/proc/$pid/limits</code>. You can read this file to check whether a max file limit value is being set properly.
=== Other limit directives ===
{{Highlight
| code = Directive        ulimit equivalent    Unit
LimitCPU=        ulimit -t            Seconds     
LimitFSIZE=      ulimit -f            Bytes
LimitDATA=      ulimit -d            Bytes
LimitSTACK=      ulimit -s            Bytes
LimitCORE=      ulimit -c            Bytes
LimitRSS=        ulimit -m            Bytes
LimitNOFILE=    ulimit -n            Number of File Descriptors
LimitAS=        ulimit -v            Bytes
LimitNPROC=      ulimit -u            Number of Processes
LimitMEMLOCK=    ulimit -l            Bytes
LimitLOCKS=      ulimit -x            Number of Locks
LimitSIGPENDING= ulimit -i            Number of Queued Signals
LimitMSGQUEUE=  ulimit -q            Bytes
LimitNICE=      ulimit -e            Nice Level
LimitRTPRIO=    ulimit -r            Realtime Priority 
LimitRTTIME=    No equivalent
| lang = text
| lang = text
}}
}}

Revision as of 00:22, 5 October 2022

Linux allows you to control how many opened file descriptors a user can have at any one time.

System-wide settings

To show the number of files opened on a system, run:

Description Command
Read the current file descriptor numbers. 3 values are:
  • Current number of opened files
  • Number of free allocated file descriptors
  • Maximum number of file descriptors for the whole system
 
# cat /proc/sys/fs/file-nr
7552    0       9223372036854775807
Maximum number of file descriptiors. 
# cat /proc/sys/fs/file-max
9223372036854775807
Set the number of file descriptors 
## Via /proc
# cat 99999 > /proc/sys/fs/file-nr

## Via sysctl
# sysctl -w fs.file-max=99999

## Edit /etc/sysctl.conf so it applies on boot
# echo "fs.file-max = 99999" >> /etc/sysctl.conf

User session setting

On typical Linux distributions using PAM, user's session's limits are controlled by editing /etc/security/limits.conf and are applied by the pam_limits.so module (Enabled when using session required pam_limits.so in the system-auth config)

A user can check their current session's limits with ulimit on the bash shell.

Description What
Max number of files opened 
# ulimit -n
1024

## Soft limit
# ulimit -Sn
1024

## Hard limit
# ulimit -Hn
1024
This tells you that the maximum number of files you can have in this session is 1024 files.

To change a user's soft and hard file limits, add the following lines: 

## In /etc/security/limits.conf
username soft nofile 16384
username hard nofile 32768

The user will have to re-login for these new limits to be applied.

Systemd service

Services that are started by systemd will not adhere to the limits specified in the /etc/security/limits.conf file (because those are set by PAM?). You will have to change the service file and specify the limits there instead.

For example: 

[Unit]
Description=Galaxy Web Service
After=network.target

[Service]
Type=simple
User=galaxy
ExecStart=/galaxy/run.sh
# ExecStop=/galaxy/run.sh stop
# ExecReload=/galaxy/run.sh restart
Restart=always

LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

Checking if limits are set properly

Each process's limits are available under /proc/$pid/limits. You can read this file to check whether a max file limit value is being set properly.

Other limit directives

Directive        ulimit equivalent     Unit
LimitCPU=        ulimit -t             Seconds      
LimitFSIZE=      ulimit -f             Bytes
LimitDATA=       ulimit -d             Bytes
LimitSTACK=      ulimit -s             Bytes
LimitCORE=       ulimit -c             Bytes
LimitRSS=        ulimit -m             Bytes
LimitNOFILE=     ulimit -n             Number of File Descriptors 
LimitAS=         ulimit -v             Bytes
LimitNPROC=      ulimit -u             Number of Processes 
LimitMEMLOCK=    ulimit -l             Bytes
LimitLOCKS=      ulimit -x             Number of Locks 
LimitSIGPENDING= ulimit -i             Number of Queued Signals 
LimitMSGQUEUE=   ulimit -q             Bytes
LimitNICE=       ulimit -e             Nice Level 
LimitRTPRIO=     ulimit -r             Realtime Priority  
LimitRTTIME=     No equivalent

See Also

Various Linux Notes
Linux Tools and Utilites
Package Management
Linux Distributions
Networking
Containers