Number of Files Opened: Difference between revisions

From Leo's Notes
This page was last edited on 5 October 2022, at 00:25.
← Older edit
VisualWikitext
No edit summary
No edit summary
 
(10 intermediate revisions by the same user not shown)
Line 1: Line 1:
Linux allows you to control how many opened file descriptors a user can have at any one time.


== System-Wide Settings ==
== System-wide settings ==


To show the number of files opened on a system, run:
To show the number of files opened on a system, run:
# cat /proc/sys/fs/file-nr
{| class="wikitable"
1536 0 793120
!Description
!Command
|-
|Read the current file descriptor numbers. 3 values are:


The fields are:
*Current number of opened files
# Current number of opened files
* Number of free allocated file descriptors
# Number of free allocated file descriptors
* Maximum number of file descriptors for the whole system
# Maximum number of file descriptors for the whole system
|{{highlight|lang=terminal|code=
# cat /proc/sys/fs/file-nr
7552    0      9223372036854775807
}}
|-
|Maximum number of file descriptiors.
|{{Highlight
| code = # cat /proc/sys/fs/file-max
9223372036854775807
| lang = terminal
}}
|-
|Set the number of file descriptors
|{{highlight|lang=terminal|code=
## Via /proc
# cat 99999 > /proc/sys/fs/file-nr


You may change this value by:
## Via sysctl
# cat <new value> > /proc/sys/fs/file-nr
# sysctl -w fs.file-max=99999
# Edit <code>/etc/sysctl.conf</code> so that there is a line
# fs.file-max = <new value>


== Per-User Settings ==
## Edit /etc/sysctl.conf so it applies on boot
# echo "fs.file-max = 99999" >> /etc/sysctl.conf
}}
|}


You can see the limits set by running <code>ulimit</code> on the bash shell.
== User session setting ==
# ulimit -n
On typical Linux distributions using PAM, user's session's limits are controlled by editing {{code|/etc/security/limits.conf}} and are applied by the <code>pam_limits.so</code> module (Enabled when using <code>session required pam_limits.so</code> in the system-auth config)


You can also see the per-user limits defined in <code>/etc/security/limits.conf</code>:
A user can check their current session's limits with {{code|ulimit}} on the bash shell.
# cat /etc/security/limits.conf
{| class="wikitable"
!Description
!What
|-
|Max number of files opened
|{{highlight
| lang = terminal
| code = # ulimit -n
1024


## Soft limit
# ulimit -Sn
1024


http://blog.thecodingmachine.com/content/solving-too-many-open-files-exception-red5-or-any-other-application
## Hard limit
# ulimit -Hn
1024
}}This tells you that the maximum number of files you can have in this session is 1024 files.
|-
|Temporarily set a new limit
|{{Highlight
| code = # ulimit -n 65536
| lang = terminal
}}
Sets both the hard and soft limit for the shell's session. Cannot exceed the limits defined in <code>/etc/security/limits.conf</code>.
|}
To change a user's soft and hard file limits, add the following lines to <code>/etc/security/limits.conf</code>:


{{Highlight
| code = ## In /etc/security/limits.conf
username soft nofile 16384
username hard nofile 32768
| lang = text
}}The user will have to re-login for these new limits to be applied.


== Systemd service ==
Services that are started by systemd will not adhere to the limits specified in the <code>/etc/security/limits.conf</code> file (because those are set by PAM?). You will have to change the service file and specify the limits there instead.


[[Category:Linux]]
For example:
{{Highlight
| code = [Unit]
Description=Galaxy Web Service
After=network.target
 
[Service]
Type=simple
User=galaxy
ExecStart=/galaxy/run.sh
# ExecStop=/galaxy/run.sh stop
# ExecReload=/galaxy/run.sh restart
Restart=always
 
LimitNOFILE=65536
 
[Install]
WantedBy=multi-user.target
| lang = text
}}
 
=== Checking if limits are set properly ===
Each process's limits are available under <code>/proc/$pid/limits</code>. You can read this file to check whether a max file limit value is being set properly.
 
=== Other limit directives ===
{{Highlight
| code = Directive        ulimit equivalent    Unit
LimitCPU=        ulimit -t            Seconds     
LimitFSIZE=      ulimit -f            Bytes
LimitDATA=      ulimit -d            Bytes
LimitSTACK=      ulimit -s            Bytes
LimitCORE=      ulimit -c            Bytes
LimitRSS=        ulimit -m            Bytes
LimitNOFILE=    ulimit -n            Number of File Descriptors
LimitAS=        ulimit -v            Bytes
LimitNPROC=      ulimit -u            Number of Processes
LimitMEMLOCK=    ulimit -l            Bytes
LimitLOCKS=      ulimit -x            Number of Locks
LimitSIGPENDING= ulimit -i            Number of Queued Signals
LimitMSGQUEUE=  ulimit -q            Bytes
LimitNICE=      ulimit -e            Nice Level
LimitRTPRIO=    ulimit -r            Realtime Priority 
LimitRTTIME=    No equivalent
| lang = text
}}
 
{{Navbox Linux}}[[Category:Linux]]

Latest revision as of 00:25, 5 October 2022

Linux allows you to control how many opened file descriptors a user can have at any one time.

System-wide settings

To show the number of files opened on a system, run:

Description Command
Read the current file descriptor numbers. 3 values are:
  • Current number of opened files
  • Number of free allocated file descriptors
  • Maximum number of file descriptors for the whole system
 
# cat /proc/sys/fs/file-nr
7552    0       9223372036854775807
Maximum number of file descriptiors. 
# cat /proc/sys/fs/file-max
9223372036854775807
Set the number of file descriptors 
## Via /proc
# cat 99999 > /proc/sys/fs/file-nr

## Via sysctl
# sysctl -w fs.file-max=99999

## Edit /etc/sysctl.conf so it applies on boot
# echo "fs.file-max = 99999" >> /etc/sysctl.conf

User session setting

On typical Linux distributions using PAM, user's session's limits are controlled by editing /etc/security/limits.conf and are applied by the pam_limits.so module (Enabled when using session required pam_limits.so in the system-auth config)

A user can check their current session's limits with ulimit on the bash shell.

Description What
Max number of files opened 
# ulimit -n
1024

## Soft limit
# ulimit -Sn
1024

## Hard limit
# ulimit -Hn
1024
This tells you that the maximum number of files you can have in this session is 1024 files.
Temporarily set a new limit 
# ulimit -n 65536

Sets both the hard and soft limit for the shell's session. Cannot exceed the limits defined in /etc/security/limits.conf.

To change a user's soft and hard file limits, add the following lines to /etc/security/limits.conf: 

## In /etc/security/limits.conf
username soft nofile 16384
username hard nofile 32768

The user will have to re-login for these new limits to be applied.

Systemd service

Services that are started by systemd will not adhere to the limits specified in the /etc/security/limits.conf file (because those are set by PAM?). You will have to change the service file and specify the limits there instead.

For example: 

[Unit]
Description=Galaxy Web Service
After=network.target

[Service]
Type=simple
User=galaxy
ExecStart=/galaxy/run.sh
# ExecStop=/galaxy/run.sh stop
# ExecReload=/galaxy/run.sh restart
Restart=always

LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

Checking if limits are set properly

Each process's limits are available under /proc/$pid/limits. You can read this file to check whether a max file limit value is being set properly.

Other limit directives

Directive        ulimit equivalent     Unit
LimitCPU=        ulimit -t             Seconds      
LimitFSIZE=      ulimit -f             Bytes
LimitDATA=       ulimit -d             Bytes
LimitSTACK=      ulimit -s             Bytes
LimitCORE=       ulimit -c             Bytes
LimitRSS=        ulimit -m             Bytes
LimitNOFILE=     ulimit -n             Number of File Descriptors 
LimitAS=         ulimit -v             Bytes
LimitNPROC=      ulimit -u             Number of Processes 
LimitMEMLOCK=    ulimit -l             Bytes
LimitLOCKS=      ulimit -x             Number of Locks 
LimitSIGPENDING= ulimit -i             Number of Queued Signals 
LimitMSGQUEUE=   ulimit -q             Bytes
LimitNICE=       ulimit -e             Nice Level 
LimitRTPRIO=     ulimit -r             Realtime Priority  
LimitRTTIME=     No equivalent
Various Linux Notes
Linux Tools and Utilites
Package Management
Linux Distributions
Networking
Containers