error: fork: Resource temporarily unavailable

This document (000020612) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server for SAP Applications 15
SUSE Linux Enterprise Server for SAP Applications 12
SUSE Linux Enterprise High Availability Extension 15
SUSE Linux Enterprise High Availability Extension 12

Situation

System hangs because it cannot spawn new child processes.
Errors found in /var/log/messages.
fork failed: Resource temporarily unavailable (errno = 11)
fatal: fork of unprivileged child failed
error: fork: Resource temporarily unavailable

If crash dump was captured, the crash logs show the following where the overall number of processes is close to pid_max configured.
crash> ps -S
  RU: 14
  IN: 65269
  WA: 1
  ZO: 3
crash>

The sar log files show that the system is close to maximum of allowed PIDs, 65536.
00:40:01      runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-15   blocked
...
02:50:01            0     65279      0.57      1.05      1.40         0
03:10:01            0     65279      0.89      1.48      1.51         0
...

pid_max configured as below.
# cat /proc/sys/kernel/pid_max
65536

Resolution

Increase the kernel variable pid_max to 131072, and monitor the system behavior.

For a temporary change, do the following:
sysctl -w kernel.pid_max=131072

For a permanent change, kernel.pid_max=131072 entry must be inserted in /etc/sysctl.conf file.

* The pid_max value depends much on the system load, thus the value must be set accordingly.

Cause

System hangs because the maximum number of processes has been reached and is not able spawn new child processes.

Additional Information

The maximum value allowed on a 64 bit system is approximately 4 million.
More info regarding pid_max could be found on 'man 5 proc'
/proc/sys/kernel/pid_max (since Linux 2.5.34)

This file specifies the value at which PIDs wrap around (i.e., the value in this file is one greater than the maximum PID). PIDs greater than this value are not allocated; thus, the value in this file also acts  as  a system-wide  limit  on  the  total number of processes and threads.  The default value for this file, 32768, results in the same range of PIDs as on earlier kernels.  On 32-bit platforms, 32768 is the maximum value for pid_max.  On 64-bit systems, pid_max can be set to any value up to 2^22 (PID_MAX_LIMIT, approximately 4 million).

Possible reasons for the errors include:
1. There is a misbehaving service or process running, consuming more resources than expected.
2. The system was not able to create new processes, because of the limits set for proc in /etc/security/limits.conf.
3. The system ran out of memory and new processes were unable to start because they could not allocate memory.
4. There is not an available ID to assign to the new process. A unique value less than kernel.pid_max must be available.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:000020612
  • Creation Date: 14-Mar-2022
  • Modified Date:16-Mar-2022
    • SUSE Linux Enterprise Server
    • SUSE Linux Enterprise Server for SAP Applications

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback@suse.com

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

Join Our Community

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.


SUSE Customer Support Quick Reference Guide SUSE Technical Support Handbook Update Advisories
Support FAQ

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.

Go to Customer Center