System lock up after XFS reporting possible memory allocation deadlock size 4194320 in kmem_alloc (mode:0x2400240)

This document (7023344) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 15
SUSE Linux Enterprise Server 12 Service Pack 3 (SLES 12 SP3)
SUSE Linux Enterprise Server 12 Service Pack 2 (SLES 12 SP2)
SUSE Linux Enterprise Server 12 Service Pack 1 (SLES 12 SP1)

Situation

While observing a high fragmentation on a XFS filesystem, the kernel starts to report messages similar to:

2018-06-25T15:21:03.532830+00:00 kernel: [8235358.733050] XFS: SubmitThread-DA(56017) possible memory allocation deadlock size 4194320 in kmem_alloc (mode:0x2400240)

and after a while locks up.

Resolution

Cause

In memory data structures associated with a fragmented file (e.g. due to suboptimal use of Direct I/O (DIO) get too big. As the number of fragments grow kmem_alloc is unable to allocate more than 4MB contiguous memory for them.

Additional Information

When using DIO all heuristics that XFS employs to optimize its IO operations are bypassed. Thus users need to carefully understand the implications of DIO and care should be taken to avoid excessive file fragmentation (i.e. using falloc to preallocate large contiguous regions of a file). Thus please understand, the patches added to the XFS code only address the hang situation, not the file fragmentation.

To show the fragmentation status of a given filesystem:
    
xfs_db -c frag -r <dev>

To start a defragmentation:
   
xfs_fsr -v <dev>
       
For further details please see "man xfs_fsr" and "man xfs_db"

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:7023344
  • Creation Date: 11-Sep-2018
  • Modified Date:03-Mar-2020
    • SUSE Linux Enterprise Server

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback@suse.com

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

Join Our Community

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.


SUSE Customer Support Quick Reference Guide SUSE Technical Support Handbook Update Advisories
Support FAQ

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.

Go to Customer Center