The sshd service easily reaches the TasksMax limit after disabling "UsePAM" in sshd_config
This document (000020981) is provided subject to the disclaimer at the end of this document.
Environment
SUSE Linux Enterprise Server 15 (All Service Packs)
SUSE Linux Enterprise Server 12 (All Service Packs)
SUSE Linux Enterprise Server for SAP Applications 15 (All Service packs)
SUSE Linux Enterprise Server for SAP Applications 12 (All Service packs)
SUSE Linux Enterprise Server 12 (All Service Packs)
SUSE Linux Enterprise Server for SAP Applications 15 (All Service packs)
SUSE Linux Enterprise Server for SAP Applications 12 (All Service packs)
Situation
An administrator has remarked out "UsePAM" in /etc/ssh/sshd_config (or set "UsePAM no"). After running for a period of time, more and more tasks are counted under the slice of sshd.service. Eventually, sshd.service reaches its Tasks limit:
New ssh connections will be rejected and sshd.service fails to restart.
# /bin/systemctl status sshd.service ● sshd.service - OpenSSH Daemon Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: disabled) Main PID: 5246 (sshd) Tasks: 65536(limit: 65536) Memory: 48.5G CPU: 6.506s CGroup: /system.slice/sshd.service ├─ 4352 app-daemon -d ├─ 5092 sshd: cmhop [priv] ├─ 5188 sshd: cmhop@pts/0 ├─ 5191 -bash sshd-gen-keys-start[10394]: Checking for missing server keys in /etc/ssh... sshd-gen-keys-start[10394]: /usr/sbin/sshd-gen-keys-start: fork: retry: No child processes
New ssh connections will be rejected and sshd.service fails to restart.
Resolution
Raise the TasksMax limit for sshd.service.
In a default configuration, most SLES 12 and 15 installations have no limit on sshd.service tasks. However, if administrators have set their own limits, then this problem may be encountered. The limit of 65536 shown above in the "Situation" section is not the default limit.
Normally, the original limit is set to "infinity" within /usr/lib/systemd/system/sshd.service and that file should not be modified (except by installation of a different openssh package version).
The more common and appropriate location to customize this setting would be within the directory:
/etc/systemd/system/sshd.service.d/
Within that directory, one or more files may have been created and may hold settings to override the original configuration. These are called "drop in" files. For example:
If UsePAM has been disabled in /etc/ssh/sshd_config, TasksMax will likely need to be set very high (or to "infinity"). Of course, another potential solution is to enable "UsePam yes" again.
In a default configuration, most SLES 12 and 15 installations have no limit on sshd.service tasks. However, if administrators have set their own limits, then this problem may be encountered. The limit of 65536 shown above in the "Situation" section is not the default limit.
Normally, the original limit is set to "infinity" within /usr/lib/systemd/system/sshd.service and that file should not be modified (except by installation of a different openssh package version).
sles15# grep TasksMax /usr/lib/systemd/system/sshd.service TasksMax=infinitySome administrators may have modified this anyway, so it should be checked.
The more common and appropriate location to customize this setting would be within the directory:
/etc/systemd/system/sshd.service.d/
Within that directory, one or more files may have been created and may hold settings to override the original configuration. These are called "drop in" files. For example:
sles15:/ grep -H TasksMax /etc/systemd/system/sshd.service.d/* /etc/systemd/system/sshd.service.d/50-TasksMax.conf:TasksMax=65536If a limit is found within a certain file, that file can be edited and the limit raised.
If UsePAM has been disabled in /etc/ssh/sshd_config, TasksMax will likely need to be set very high (or to "infinity"). Of course, another potential solution is to enable "UsePam yes" again.
Cause
When UsePAM is disabled, "pam_systemd.so" is not loaded when sshd initiates a new ssh client process. Therefore, no new systemd session will be created. All ssh connections and all programs run within them will be counted as tasks under the single slice of sshd.service.
Disabling "pam_systemd.so" in /etc/pam.d/common-session will cause the same problem.
Disabling "pam_systemd.so" in /etc/pam.d/common-session will cause the same problem.
Additional Information
The currently effective setting of TasksMax for sshd.service can be found with the following command. The default configuration of "infinity" results in an extremely high value, as shown here:
Some very early SLES 12 distributions (for example SLES 12 SP0, maybe SP1), especially without maintenance updates applied, may show lower TasksMax limits for sshd.service and many other services. Most TasksMax limits were too low when SLES 12 first shipped, but that was soon rectified.
sles15:/ # systemctl show 'sshd.service' | grep TasksMax TasksMax=18446744073709551615
Some very early SLES 12 distributions (for example SLES 12 SP0, maybe SP1), especially without maintenance updates applied, may show lower TasksMax limits for sshd.service and many other services. Most TasksMax limits were too low when SLES 12 first shipped, but that was soon rectified.
Disclaimer
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000020981
- Creation Date: 21-Feb-2023
- Modified Date:07-Mar-2023
-
- SUSE Linux Enterprise Server
- SUSE Linux Enterprise Server for SAP Applications
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com
