System does not boot after extending btrfs root filesystem

This document (000019686) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 12 Service Pack 5 - System Z (s390x)
SUSE Linux Enterprise Server 12 Service Pack 4 - System Z (s390x)

Situation

After adding a disk volume to the volume group of the root filesystem, or in case of btrfs, to the root filesystem itself, the root filesystem cannot be mounted on next IPL.
Systemd endlessly prints messages similar to:
"A start job is running for dev-disk...ab12cdef.device (60s / no limit)."

Example:
btrfs device add /dev/dasdb1 /

If you do this and have not rebooted, see TID 000019088 - System does not boot anymore after extending root filesystem for list of commands to run BEFORE reboot.

Resolution

If you have rebooted and the system will not boot the following are steps to recover the SLES VMs running on s390, IBM Z.

In short, the recovery process, is to run "grub2-install" but from a rescue boot, not from mounting on another VM.
Then you have to copy the new initrd from /boot/zipl/ to /boot/

There are variables that will need to be adjusted for each environment.
It is assumed you know how to use the IBM tools to adjust parmfiles, minidisk and IPL the system.

We assume you know how to boot the VM so it can start an FTP or HTTP install,
we'll add the "rescue" flag to the boot parameters instead of completing a normal install.

Here is an example parmfile for that:
InstNetDev=osa OsaInterface=qdio Layer2=1 PortNo=0 OSAHWAddr=
HostIP=10.1.1.10/20 Hostname=sles12sp5.domain.com
Gateway=10.1.1.1 Nameserver=10.1.1.1 Domain=domain.com
ReadChannel=0.0.0600 WriteChannel=0.0.0601 DataChannel=0.0.0602
install=ftp://ftp-install.server.com/install/SLP/SLE-12-SP5-Server-GM/s390x/DVD1/
ssh=1 sshpassword=sshpasswd rescue=1

You will need the SLES OS s390 ISO.
The SLES12-SP5 s390 ISO is:
SLE-12-SP5-Server-DVD-s390x-GM-DVD1.iso

You will need to put these 4 files from the SLES12-SP5 s390 ISO on the "a" disk.
The files on the ISO are:
boot/s390x/linux
boot/s390x/initrd
boot/s390x/sles12.exec
boot/s390x/parmfile

(With the parmfile edited to what is posted above).
The parmfile will need to be changed and customized for your environment.

The files are punched to the virtual card reader and then IPL'd
(booted) from the reader.

to start the VM run:
sles12 exec

Once you're booted into rescue mode we want to name
the devices what they are actually named in the running OS.
Typically that would be 0100 and 0101.
chzdev -e 0.0.0100
chzdev -e 0.0.0101

lsdasd (to confirm)
btrfs filesystem show / (to confirm both disks)
mount /dev/dasda2 /mnt/
mount /dev/dasda1 /mnt/boot/zipl
mount --rbind /dev /mnt/dev
mount --rbind /proc /mnt/proc
mount --rbind /sys /mnt/sys
chroot /mnt/


Do not run "mkinitrd" or any of the other commands.

Only run:
grub2-install


Then we need to copy the NEW initrd that was created in /boot/zipl/ to /boot/

Using "ls", you can check the dates, the initrd in /boot/zipl should be the current date:
(Depending on kernel version)
ls -la /boot/zipl/initrd-4.12.14-122.12-default

This one should be old:
ls -la /boot/initrd-4.12.14-122.12-default

Then:
cp /boot/zipl/initrd-4.12.14-122.12-default /boot/initrd-4.12.14-122.12-default
(Change the kernel numbers to match the version you are running.)

Exit chroot and reboot:
exit
umount /mnt/sys
umount /mnt/proc
umount /mnt/dev
umount /mnt/boot/zip/
umount /mnt/
reboot

Then the VM should boot normally.

Cause

Grub2 and the initrd will only set disk devices online that are needed to mount the root filesystem to avoid long waiting times when a large amount of disks are visible to the system. Thus, both need to be updated to contain the information that an additional disk is required. This affects only filesystems mandatory to start the installed system, for filesystems that get mounted afterwards this is not necessary.
 

Additional Information

Related to TID 000019088 .

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:000019686
  • Creation Date: 20-Aug-2020
  • Modified Date:20-Aug-2020
    • SUSE Linux Enterprise Server

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback@suse.com

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

Join Our Community

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.


SUSE Customer Support Quick Reference Guide SUSE Technical Support Handbook Update Advisories
Support FAQ

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.

Go to Customer Center