Filesystem goes read-only without the underlying storage reporting errors

This document (3735930) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 10
SUSE Linux Enterprise Desktop 10

Situation

Filesystem (ext3, reiserfs or udf) is remounted read-only by the kernel.
No errors are reported from the storage layer underlying the affected filesystem.

Resolution

For SLE 10 (SUSE Linux Enterprise 10), this issue has been resolved for the ext3 filesystem as of kernel update 2.6.16.27-0.9, released Feb 23, 2007. For the reiserfs and udf filesystems, this issue is fixed as of the release of Service Pack 1 (kernel 2.6.16.46-0.12).

Additional Information

When the filesystem code detects the occurrence of an error from which recovery without an fsck is not possible, it changes state to read-only to prevent further corruption.

Normally, this behavior occurs when the underlying storage layer (like SCSI) reported an error when the filesystem code was trying to write out vital data. Errors or warnings reported by the underlying storage layer will precede the messages from the filesystem layer and the issue can only be addressed by fixing the hardware and/or configuration problem with the storage layer.

With a system under heavy load, a bug in the read-ahead optimization part of the ext3, reiserfs and udf code could be triggered in which it mistook a temporary error condition from the storage layer ("please retry again later") for an unrecoverable error.In this case, only the filesystem layer reports errors; these errors are not preceded by errors or warnings reported by the underlying storage layer.

In the case of ext3, messages similar to the following arelogged:
ERROR: EXT3-fs error (device sda1): ext3_readdir: directory #133120 containsa hole at offset 4096
ERROR: Aborting journal on device sda1.
ERROR: ext3_abort called.
ERROR: EXT3-fs abort (device sda1):
ERROR: ext3_journal_start: Detected aborted journal
ERROR: Remounting filesystem read-only

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.