kernel: nfsd: too many open TCP sockets, consider increasing the number of threads

This document (7010903) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 15
SUSE Linux Enterprise Server 12
SUSE Linux Enterprise Server 11
SUSE Linux Enterprise Server 10

Situation

A busy NFS server may occasionally log the following message in /var/log/messages:
 
kernel: nfsd: too many open TCP sockets, consider increasing the number of nfsd threads

Resolution

The threads in question are set in /etc/sysconfig/nfs:
USE_KERNEL_NFSD_NUMBER="4"
 
The default is 4 but can be raised to allow nfsd to handle more requests at once.  Based on experience with many cases, SUSE support gives the following guidelines:

Boosting the number of threads (even dramatically) is not likely to cause any negative side effects unless the machine in question is already struggling to keep up with workload even before any NFS work is introduced.  Therefore, it is safe to be generous with this setting.

When putting any NFS Server into enterprise production, it would be good to boost this setting to at least 16.  This will give it more room for load than the default of 4.

Then, if NFS client performance problems are seen or suspected, experiment with values in the range of 32 to 128.  Something within this range will usually prove to be both safe and sufficient for busy enterprise servers.  In some cases of extreme load, going even higher than 128 may be needed.


After making a change, the nfs server daemon can be restarted with:

ON SLES 12 or 15:
systemctl restart nfsserver

On SLES 10 or 11:
rcnfsserver restart

Cause

If the NFS Server has a lot of requests coming in, the default 4 threads will not likely keep up with the workload.  The message in the "Situation" section above is generated by some very basic rule-of-thumb checking, and is not a great indicator of when performance problems might begin to happen.  It does not issue the warning based on actual work being done, but only based on the number of NFS connections present.

The message is given when connections to the NFS server's port 2049 are greater than:

[(number of threads + 3) * 20]

For example, at the default of 4, the message will be logged if connections to port 2049 exceed 140.  However, there could be many connection present with each doing very little work, or there could fewer connections each doing a tremendous amount of work. Performance problems may exist even without the warning being logged.
 
To decide how many threads are need to merely avoid the warning, it can sometimes be helpful to manually check the number of nfsd connections, especially over time or at peak work times.  To do so, give the following command (at a terminal on the NFS Server).
 
netstat -nt | grep -c :2049 | wc -l
 
Then plug the trended count or peak count into the formula above and calculate the number of threads which would be needed to satisfy the algorithm.

However, even before reaching the number of connections which the formula indicates, performance problems related to the number of threads will often exist.  Therefore, the recommendations given in the 'Resolution" section above are typically higher than would be indicated by the formula.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:7010903
  • Creation Date: 10-Oct-2012
  • Modified Date:01-Oct-2021
    • SUSE Linux Enterprise Server

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

Join Our Community

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.


SUSE Customer Support Quick Reference Guide SUSE Technical Support Handbook Update Advisories
Support FAQ

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.

Go to Customer Center