NFS clients getting 'permission denied', even when ownership and permissions are correct

This document (7023210) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 11 Service Pack 4 (SLES 11 SP4)
 

Situation

Machines acting as NFS clients would mount NFS shares successfully and then function correctly for a time, but occasionally begin getting "permission denied" errors when attempting operations within the NFS mounted file system, even on actions as simple as "ls".  File system ownership and permission are correct, for these users to access the files / directories in question.
 
Looking at packet traces of NFS activities during these times, the RPC layer returns error.  Wireshark displays these errors as:
 
Reject State: AUTH_ERROR (1)
Auth State: bad credential (seal broken) (1)
 
The problem comes and goes.
 
While the problem is occurring, new attempts to mount something from the NFS server also fail with "permission denied."

Resolution

The NFS server machine was having intermittent access to DNS resolution.  Therefore, the RPC layer could not always verify the host names which belonged to the client IP addresses.  Forward and backward name resolution is important to NFS security.  An NFS Server should be able to find A records (for name to address resolution) and PTR records (for address to name resolution) for any NFS client machine which will be accessing NFS shares.
 
In the case in question, there was a DNS server which would not always answer requests.  Other DNS servers were functioning reliably.  The administrator took the faulty DNS server out of service, and the problem was corrected.
 
If correction of DNS behavior or contents is not immediately feasible, then adding entries to the NFS Server's /etc/hosts file, for the NFS client machines in question, will also resolve the problem.

Additional Information

Most cases of missing DNS information are not intermittent and therefore a DNS failure (or lack of information) simply causes the inability to accomplish an NFS mount operation.  Since there are limited reasons that a NFS Server would deny a client's request to mount, most cases of missing DNS information are discovered relatively quickly.
 
However, when the DNS problem is intermittent, clients may successful mount an NFS share, but much later may get "permission denied" while trying to use the already-mounted NFS file systems.  This scenario is a bit harder to recognize, but the most important clue is whether the RPC layer is returning the auth and credential errors mentioned above in the "Situation" section.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:7023210
  • Creation Date: 25-Jul-2018
  • Modified Date:03-Mar-2020
    • SUSE Linux Enterprise Server

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback@suse.com

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

Join Our Community

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.


SUSE Customer Support Quick Reference Guide SUSE Technical Support Handbook Update Advisories
Support FAQ

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.

Go to Customer Center