multipath system unable to boot after installing dracut-037-98.2.x86_64

This document (7020912) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 12 Service Pack 1 (SLES 12 SP1)
SUSE Linux Enterprise Server for SAP Applications

Situation

Applying updates to the current patch level of mirrored repositories without the required /etc/dracut.conf.d/10-mp.conf can lead to a race condition in the boot process.

After updating to the following versions:
 
dracut-037-98.2.x86_64
kernel-default-3.12.74-60.64.40.1.x86_64
kpartx-0.5.0-62.1.x86_64
multipath-tools-0.5.0-62.1.x86_64

a boot failure will result in being dropped to the emergency shell.
##Error from boot log

May 26 12:12:38 <hostname> kernel: device-mapper: multipath service-time: version
May 26 12:12:38 <hostname> multipathd[1191]: 3600c0ff000275a1178741d5901000000: l
May 26 12:12:38 <hostname> multipathd[1191]: 3600c0ff000275a1178741d5901000000: e
May 26 12:12:38 <hostname> kernel: bio: create slab <bio-1> at 1
May 26 12:12:38 <hostname> systemd-fsck[1200]: /sbin/fsck.xfs: XFS file system.
May 26 12:12:38 <hostname> systemd[1]: Started File System Check on /dev/disk/by-
May 26 12:12:38 <hostname> systemd[1]: Mounting /sysroot...
May 26 12:12:38 <hostname> kernel: bio: create slab <bio-2> at 2
May 26 12:12:38 <hostname> kernel: SGI XFS with ACLs, security attributes, realti
May 26 12:12:38 <hostname> mount[1220]: mount: /dev/sdc3 is already mounted or /s
May 26 12:12:38 <hostname> systemd[1]: [1;39msysroot.mount mount process exited, code=e
May 26 12:12:38 <hostname> systemd[1]: [1;31mFailed to mount /sysroot.
May 26 12:12:38 <hostname> systemd[1]: [1;31mDependency failed for Initrd Root File Sys
[7mlines 2969-2991
[27m
[KMay 26 12:12:38 <hostname> systemd[1]:
[1;31mDependency failed for Reload Configuration
May 26 12:12:38 <hostname> systemd[1]: Triggering OnFailure= dependencies of init
May 26 12:12:38 <hostname> systemd[1]: Triggering OnFailure= dependencies of init
May 26 12:12:38 <hostname> systemd[1]:
[1;39mUnit sysroot.mount entered failed state.
May 26 12:12:38 <hostname> systemd[1]:

Resolution

Ensure the initrd is built with the required dm-multipath modules and the rd.driver.pre=dm_multipath is present.
 
The file /etc/dracut.conf.d/10-mp.conf should exist _before_ the updates are applied to avoid the issue.
This is mentioned in the Section 15.11.2 "Enabling Multipath I/O for an Existing Root Device" in the Storage Admin Guide.

If updates were applied without the required file then you can do the following in the Emergency Shell:
       'dracut --add multipath'
in order to create a multipath-enabled initrd and try booting again.

Or  backrev the versions to:

kpartx-0.5.0-55.1.x86_64.rpm
multipath-tools-0.5.0-55.1.x86_64.rpm
dracut-037-84.1.x86_64.rpm

This will return to normal booting condition where you can do the following to ensure proper boot after updates:

1. Create the required /etc/dracut.conf.d/10-mp.conf file by entering:
     
     echo 'force_drivers+=" dm-multipath "' >/etc/dracut.conf.d/10-mp.conf
(the empty space before and after " dm-multipath " is intended)

2. Apply updates with 'zypper patch'

3. Verify that you see rd.driver.pre=dm_multipath in the "Stored Kernel Commandline" of the mkinitrd log.
The mkinitrd will be the last in the process of patching and will be on the screen when zypper finishes.
Make sure that the "rd.driver.pre=dm_multipath" is present in the initrd build before attempting to reboot.

Example snip of a good initrd build:

I: Stored kernel commandline:
I: rd.driver.pre=dm_multipath       
I:  rd.lvm.lv=vg_alfsd2p1/lv_swap
 rd.lvm.lv=rhel/swap
I: rd.driver.pre=scsi_dh_alua rd.driver.pre=scsi_dh_emc rd.driver.pre=scsi_dh_rdac

Cause

The problem is a miss-configuration of not having the required /etc/dracut.conf.d/10-mp.conf file containing
    force_drivers+=" dm-multipath "

Without it the new initrd created by the kernel update process will not have the required pre-checkers and dm-multipath.

Additional Information


Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:7020912
  • Creation Date: 13-Jun-2017
  • Modified Date:03-Mar-2020
    • SUSE Linux Enterprise Server
    • SUSE Linux Enterprise Server for SAP Applications

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback@suse.com

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

Join Our Community

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.


SUSE Customer Support Quick Reference Guide SUSE Technical Support Handbook Update Advisories
Support FAQ

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.

Go to Customer Center