Random 'permission denied', 'check lease failed' and rpc errors

SUSE Linux Enterprise Server 15 Service Pack 6

A mixture of errors are seen at the NFS clients and NFS servers.

At the NFS clients:

[Wed Sep 17 13:24:53 2025] NFS: state manager: check lease failed on NFSv4 server <ip_address>  with error 13

name@server:/sapmnt> ls -la
/bin/ls: cannot access 'file1': Permission denied
total 4
drwxr-xr-x  3 root sapsys   17 Oct 11  2021 .
drwxr-xr-x 28 root root   4096 Sep 17 16:38 ..
d?????????  ? ?    ?         ?            ? file1

At the NFS server:

2025-09-17T08:11:30.393840+00:00 <server_name> kernel: [T101906] rpc-srv/tcp: nfsd: got error -104 when sending 20 bytes - shutting down socket
2025-09-17T08:13:29.429815+00:00 <server_name> kernel: [T101906] rpc-srv/tcp: nfsd: got error -32 when sending 20 bytes - shutting down socket

The errors appear at random intervals and sometimes persist with specific client/server combinations for a few hours.

Please note that various and different issues might cause the above errors and as such, they are somewhat superficial and may occur in scenarios other than the one this document is meant to discuss.  For this specific scenario, the underlying symptom is the RPC layer is denying the NFS client's access to the NFS Server.  In Wireshark (or other packet reading tool) the RPC header shows (in part):

Remote Procedure Call, Type:Reply XID:0x<nnnnnnnn>
     Message Type: Reply (1)
     <snip>
     Reply State: denied (1)
     Reject State: AUTH_ERROR (1)
     Auth State: bad credential (seal broken) (1)

This indicates that the NFS Server does not consider the client in question to be authorized to use the NFS path in question.  It may be related to the export syntax of the path and it's trusted client specification, as well as any code or services needed to resolve the client specification from names to addresses or to resolve netgroup names to membership lists.

Making both of the following changes has been shown to avoid the issue:

At the time of creating this document, it is unclear what the exact cause of the problem is.

The problem may be related to unreliable DNS resolution and the caching of DNS lookup failures. The problem may also involve something specific to netgroups (when used for NFS client authentication), or a combination of some or all of these elements.

Should these issues be encountered, a support case can be opened using an appropriate entitlement.

In addition to providing supportconfigs from affected NFS clients and NFS servers, the following procedure should be followed and the resulting data should be collected and added to a support case:

Notes: 

The server-side errors are socket-related.
The caching of DNS resolution results related to trusted clients is controlled by a kernel cache managed by rpc.mountd
The default life time of the kernel cache entries managed by rpc.mountd is 1800 seconds (adjustable).
This scope of this issue may not be limited to SLES15 SP6.