System fails to generate a kernel crashdump and drops into bash shell after triggering kdump

This document (7015824) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 12 (ppc64le)

Situation

After installation of SLES 12 GMC on the ppc64le architecture, an attempt to trigger a kernel crashdump gives the following result:

tux:~ # echo c > /proc/sysrq-trigger
SysRq : Trigger a crash
Oops: Kernel access of bad area, sig: 11 [#1]
SMP NR_CPUS=2048 NUMA pSeries
Modules linked in: af_packet ibmveth(X) rtc_generic dm_mod xfs libcrc32c sd_mod crc_t10dif ibmvscsi(X)... 
Supported: Yes
CPU: 2 PID: 22414 Comm: bash Tainted: G              X 3.12.28-4-default #1
task: c0000003baf5b430 ti: c0000003bcf2c000 task.ti: c0000003bcf2c000
NIP: c0000000004a3634 LR: c0000000004a46d0 CTR: c0000000004a3600
REGS: c0000003bcf2f9c0 TRAP: 0300   Tainted: G              X  (3.12.28-4-default)
MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE>  CR: 42424422  XER: 00000000
CFAR: 00003fff82b40b6c DAR: 0000000000000000 DSISR: 42000000 SOFTE: 0
GPR00: c0000000004a46d0 c0000003bcf2fc40 c000000000e6e538 0000000000000063
GPR04: c000000001385b68 c000000001396340 0a72696767657220 00000000000000c3
GPR08: c000000000a1e538 0000000000000001 0000000000000000 c000000000710860
GPR12: c0000000004a3600 c000000007b81200 00000000100e0cd4 0000000000000000
GPR16: 0000000000000000 0000000000000000 0000000000000000 0000010030508210
GPR20: 00000000100b1ac0 00000000100e4d78 00000000100e4010 0000000000000000
GPR24: 0000000000000001 0000000000000000 0000000000000001 c000000000de1340
GPR28: c00000000121e190 0000000000000063 c000000000daf25c c000000000de1700
NIP [c0000000004a3634] sysrq_handle_crash+0x34/0x50
LR [c0000000004a46d0] __handle_sysrq+0x100/0x260
Call Trace:
[c0000003bcf2fc40] [0000000000000063] 0x63 (unreliable)
[c0000003bcf2fc60] [c0000000004a46d0] __handle_sysrq+0x100/0x260
[c0000003bcf2fd10] [c0000000004a4e78] write_sysrq_trigger+0x78/0xa0
[c0000003bcf2fd40] [c000000000310860] proc_reg_write+0xb0/0x110
[c0000003bcf2fd90] [c000000000284d5c] vfs_write+0xec/0x250
[c0000003bcf2fde0] [c000000000285a9c] SyS_write+0x6c/0xf0
[c0000003bcf2fe30] [c00000000000a17c] syscall_exit+0x0/0x7c
Instruction dump:
3842af38 7c0802a6 f8010010 f821ffe1 60000000 60000000 3d220020 3949a1b0
39200001 912a0000 7c0004ac 39400000 <992a0000> 38210020 e8010010 7c0803a6
---[ end trace 686660ce76bb97d7 ]---

Sending IPI to other CPUs
IPI complete
I'm in purgatory
 -> smp_release_cpus()
spinning_secondaries = 0
 <- smp_release_cpus()
 <- setup_system()
CF000012
CF000015ch
SUSE Linux
#1 SMP Thu Sep 2no ibm,pcie-link-speed-stats property
opal_async_comp_init: Opal node not found
nvram: Failed to find partition of-config, err 0
[FAILED] Failed to mount /sysroot.
See "systemctl status sysroot.mount" for details.
[DEPEND] Dependency failed for Initrd Root File System.
[DEPEND] Dependency failed for Reload Configuration from the Real Root.

Generating "/run/initramfs/rdsosreport.txt"


Entering emergency mode. Exit the shell to continue.
Type "journalctl" to view system logs.
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot
after mounting them and attach it to a bug report.


:/#

Resolution

This is a known issue that affects only the ppc64le architecture, and SUSE is working on a fix that will be published as soon as available.

However, in the meantime, it is possible to work around this issue by using the following command:

 #makedumpfile -c -d 31 /proc/vmcore /KDUMP_DIRECTORY

Where "/KDUMP_DIRECTORY" is to be replaced with the directory where the kernel crashdump will be saved.

Cause

This is due to a regression that has been introduced between RC3 and GMC.

Additional Information

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.