Number of Files Opened: Difference between revisions

From Leo's Notes
This page was last edited on 5 October 2022, at 00:25.
← Older edit
VisualWikitext
No edit summary
No edit summary
 
(One intermediate revision by the same user not shown)
Line 1: Line 1:
Linux allows you to control how many opened file descriptor an user can have at any one time.
Linux allows you to control how many opened file descriptors a user can have at any one time.


== System-Wide Settings ==
== System-wide settings ==


To show the number of files opened on a system, run:
To show the number of files opened on a system, run:
{{highlight|lang=terminal|code=
{| class="wikitable"
!Description
!Command
|-
|Read the current file descriptor numbers. 3 values are:
 
*Current number of opened files
* Number of free allocated file descriptors
* Maximum number of file descriptors for the whole system
|{{highlight|lang=terminal|code=
# cat /proc/sys/fs/file-nr
# cat /proc/sys/fs/file-nr
1536 0 793120
7552    0       9223372036854775807
}}
|-
|Maximum number of file descriptiors.
|{{Highlight
| code = # cat /proc/sys/fs/file-max
9223372036854775807
| lang = terminal
}}
}}
|-
|Set the number of file descriptors
|{{highlight|lang=terminal|code=
## Via /proc
# cat 99999 > /proc/sys/fs/file-nr


The fields are:
## Via sysctl
# Current number of opened files
# sysctl -w fs.file-max=99999
# Number of free allocated file descriptors
# Maximum number of file descriptors for the whole system


You may change this value by:
## Edit /etc/sysctl.conf so it applies on boot
{{highlight|lang=terminal|code=
# echo "fs.file-max = 99999" >> /etc/sysctl.conf
# cat <new value> > /proc/sys/fs/file-nr
## Edit {{code|/etc/sysctl.conf}} so that there is a line
## fs.file-max = <new value>
}}
}}
|}


== Per-User Settings ==
== User session setting ==
On typical Linux distributions using PAM, user's session's limits are controlled by editing {{code|/etc/security/limits.conf}} and are applied by the <code>pam_limits.so</code> module (Enabled when using <code>session required pam_limits.so</code> in the system-auth config)


You can see the limits set by running {{code|ulimit}} on the bash shell.
A user can check their current session's limits with {{code|ulimit}} on the bash shell.
{{highlight
{| class="wikitable"
!Description
!What
|-
|Max number of files opened
|{{highlight
| lang = terminal
| lang = terminal
| code = # ulimit -n
| code = # ulimit -n
1024
1024
## Soft limit
# ulimit -Sn
1024
## Hard limit
# ulimit -Hn
1024
}}This tells you that the maximum number of files you can have in this session is 1024 files.
|-
|Temporarily set a new limit
|{{Highlight
| code = # ulimit -n 65536
| lang = terminal
}}
}}
This tells you that the maximum number of files you can have in this session is 1024 files.
Sets both the hard and soft limit for the shell's session. Cannot exceed the limits defined in <code>/etc/security/limits.conf</code>.
 
|}
For normal shell login sessions, you can specify these limits in {{code|/etc/security/limits.conf}}. To change a user's soft and hard file limits, add the following lines:
To change a user's soft and hard file limits, add the following lines to <code>/etc/security/limits.conf</code>:


{{Highlight
{{Highlight
Line 38: Line 75:
username hard nofile 32768
username hard nofile 32768
| lang = text
| lang = text
}}
}}The user will have to re-login for these new limits to be applied.


== Systemd service ==
== Systemd service ==
Line 64: Line 101:
}}
}}


== See Also ==
=== Checking if limits are set properly ===
* http://blog.thecodingmachine.com/content/solving-too-many-open-files-exception-red5-or-any-other-application
Each process's limits are available under <code>/proc/$pid/limits</code>. You can read this file to check whether a max file limit value is being set properly.
 
=== Other limit directives ===
{{Highlight
| code = Directive        ulimit equivalent    Unit
LimitCPU=        ulimit -t            Seconds     
LimitFSIZE=      ulimit -f            Bytes
LimitDATA=      ulimit -d            Bytes
LimitSTACK=      ulimit -s            Bytes
LimitCORE=      ulimit -c            Bytes
LimitRSS=        ulimit -m            Bytes
LimitNOFILE=    ulimit -n            Number of File Descriptors
LimitAS=        ulimit -v            Bytes
LimitNPROC=      ulimit -u            Number of Processes
LimitMEMLOCK=    ulimit -l            Bytes
LimitLOCKS=      ulimit -x            Number of Locks
LimitSIGPENDING= ulimit -i            Number of Queued Signals
LimitMSGQUEUE=  ulimit -q            Bytes
LimitNICE=      ulimit -e            Nice Level
LimitRTPRIO=    ulimit -r            Realtime Priority 
LimitRTTIME=    No equivalent
| lang = text
}}


{{Navbox Linux}}[[Category:Linux]]
{{Navbox Linux}}[[Category:Linux]]

Latest revision as of 00:25, 5 October 2022

Linux allows you to control how many opened file descriptors a user can have at any one time.

System-wide settings

To show the number of files opened on a system, run:

Description Command
Read the current file descriptor numbers. 3 values are:
  • Current number of opened files
  • Number of free allocated file descriptors
  • Maximum number of file descriptors for the whole system
 
# cat /proc/sys/fs/file-nr
7552    0       9223372036854775807
Maximum number of file descriptiors. 
# cat /proc/sys/fs/file-max
9223372036854775807
Set the number of file descriptors 
## Via /proc
# cat 99999 > /proc/sys/fs/file-nr

## Via sysctl
# sysctl -w fs.file-max=99999

## Edit /etc/sysctl.conf so it applies on boot
# echo "fs.file-max = 99999" >> /etc/sysctl.conf

User session setting

On typical Linux distributions using PAM, user's session's limits are controlled by editing /etc/security/limits.conf and are applied by the pam_limits.so module (Enabled when using session required pam_limits.so in the system-auth config)

A user can check their current session's limits with ulimit on the bash shell.

Description What
Max number of files opened 
# ulimit -n
1024

## Soft limit
# ulimit -Sn
1024

## Hard limit
# ulimit -Hn
1024
This tells you that the maximum number of files you can have in this session is 1024 files.
Temporarily set a new limit 
# ulimit -n 65536

Sets both the hard and soft limit for the shell's session. Cannot exceed the limits defined in /etc/security/limits.conf.

To change a user's soft and hard file limits, add the following lines to /etc/security/limits.conf: 

## In /etc/security/limits.conf
username soft nofile 16384
username hard nofile 32768

The user will have to re-login for these new limits to be applied.

Systemd service

Services that are started by systemd will not adhere to the limits specified in the /etc/security/limits.conf file (because those are set by PAM?). You will have to change the service file and specify the limits there instead.

For example: 

[Unit]
Description=Galaxy Web Service
After=network.target

[Service]
Type=simple
User=galaxy
ExecStart=/galaxy/run.sh
# ExecStop=/galaxy/run.sh stop
# ExecReload=/galaxy/run.sh restart
Restart=always

LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

Checking if limits are set properly

Each process's limits are available under /proc/$pid/limits. You can read this file to check whether a max file limit value is being set properly.

Other limit directives

Directive        ulimit equivalent     Unit
LimitCPU=        ulimit -t             Seconds      
LimitFSIZE=      ulimit -f             Bytes
LimitDATA=       ulimit -d             Bytes
LimitSTACK=      ulimit -s             Bytes
LimitCORE=       ulimit -c             Bytes
LimitRSS=        ulimit -m             Bytes
LimitNOFILE=     ulimit -n             Number of File Descriptors 
LimitAS=         ulimit -v             Bytes
LimitNPROC=      ulimit -u             Number of Processes 
LimitMEMLOCK=    ulimit -l             Bytes
LimitLOCKS=      ulimit -x             Number of Locks 
LimitSIGPENDING= ulimit -i             Number of Queued Signals 
LimitMSGQUEUE=   ulimit -q             Bytes
LimitNICE=       ulimit -e             Nice Level 
LimitRTPRIO=     ulimit -r             Realtime Priority  
LimitRTTIME=     No equivalent
Various Linux Notes
Linux Tools and Utilites
Package Management
Linux Distributions
Networking
Containers