SUSE Support

Here When You Need Us

Kernel update to version 4.4.155-94.50.1 / 4.4.156-94.57 / 4.4.156-94.61.1 breaks XEN virtualization

This document (7023434) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 12 Service Pack 3 (SLES 12 SP3)

Situation

After applying one of the SLES12 SP3 kernel 

4.4.155-94.50.1
4.4.156-94.57
4.4.156-94.61.1,

XEN guests (DomU) are crashing after  a couple of minutes with stack traces similar to the following:

[ 2720.184804] ------------[ cut here ]------------
[ 2720.184871] kernel BUG at ../drivers/block/xen-blkfront.c:277!
[ 2720.184937] invalid opcode: 0000 [#1] SMP 
[ 2720.184998] Modules linked in: ip6t_REJECT nf_log_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 xt_REDIRECT nf_nat_redirect ipt_REJECT xt_tcpudp nf_log_ipv4 nf_log_common xt_LOG xt_limit xt_comment ip6table_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_conntrack iptable_filter af_packet 8021q garp mrp stp llc ip6_tables nf_nat_pptp nf_nat_proto_gre nf_conntrack_pptp iscsi_ibft nf_conntrack_proto_gre iscsi_boot_sysfs nf_nat_tftp nf_conntrack_tftp nf_nat_ftp nf_nat nf_conntrack_ftp xt_state nf_conntrack ip_tables x_tables intel_rapl sb_edac edac_core crct10dif_pclmul xen_fbfront crc32_pclmul syscopyarea crc32c_intel sysfillrect sysimgblt fb_sys_fops ghash_clmulni_intel joydev drbg xen_netfront ansi_cprng xen_kbdfront aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd pcspkr ext4 crc16 jbd2 mbcache xen_blkfront dm_mirror dm_region_hash dm_log sg dm_multipath dm_mod scsi_dh_rdac scsi_dh_emc scsi_dh_alua scsi_mod autofs4
[ 2720.186404] Supported: Yes
[ 2720.186438] CPU: 2 PID: 5849 Comm: file Not tainted 4.4.155-94.50-default #1
[ 2720.186506] task: ffff880002e3cc80 ti: ffff8800ffbc0000 task.ti: ffff8800ffbc0000
[ 2720.186576] RIP: e030:[<ffffffffa00be685>]  [<ffffffffa00be685>] get_free_grant+0x55/0x60 [xen_blkfront]
[ 2720.186683] RSP: e02b:ffff8800ffbc36d8  EFLAGS: 00010046
[ 2720.186738] RAX: ffff880002c330f8 RBX: ffff8800033c06d0 RCX: ffff8800ffbc37a8
[ 2720.186805] RDX: ffff880002c330f8 RSI: 0000000000000000 RDI: ffff880002c20000
[ 2720.186872] RBP: ffff8800ffbc37a8 R08: ffff8800ffbc37a8 R09: 0000000000001000
[ 2720.186933] R10: 0000000000000000 R11: 00000000000000bc R12: 0000000000001000
[ 2720.186995] R13: ffff880002c20000 R14: ffff880002c20358 R15: ffff8800ffbc37c8
[ 2720.187076] FS:  00007ff74f1d0700(0000) GS:ffff8801f5d00000(0000) knlGS:0000000000000000
[ 2720.187146] CS:  e033 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2720.187198] CR2: 00007ff74e51d098 CR3: 0000000002ff6000 CR4: 0000000000042660
[ 2720.187266] Stack:
[ 2720.187291]  ffffffffa00c4127 0000000000000019 000000000013c474 ffff8800031ce220
[ 2720.187385]  ffff8800031f04c0 ffff8801f3465200 0000000000f5c3a0 0000000000001000
[ 2720.188013]  ffff8800ffbc37a8 ffffffffa00c40d0 00000000001eb875 0000000000001000
[ 2720.188013] Call Trace:
[ 2720.188013]  [<ffffffffa00c4127>] blkif_setup_rw_req_grant+0x57/0x4c5 [xen_blkfront]
[ 2720.188013]  [<ffffffff81404fa6>] gnttab_foreach_grant_in_range+0x66/0x100
[ 2720.188013]  [<ffffffffa00c0112>] blkif_queue_rq+0x3c2/0x6e0 [xen_blkfront]
[ 2720.188013]  [<ffffffff81306065>] blk_mq_dispatch_rq_list+0xd5/0x1e0
[ 2720.188013]  [<ffffffff8130629e>] blk_mq_process_rq_list+0x12e/0x150
[ 2720.188013]  [<ffffffff8130647b>] __blk_mq_delay_run_hw_queue+0xcb/0xd0
[ 2720.188013]  [<ffffffff81307ede>] blk_mq_flush_plug_list+0x11e/0x130
[ 2720.188013]  [<ffffffff812fc06c>] blk_flush_plug_list+0xcc/0x210
[ 2720.188013]  [<ffffffff8130720c>] blk_sq_make_request+0x16c/0x480
[ 2720.188013]  [<ffffffff812fa6a4>] generic_make_request+0x104/0x2e0
[ 2720.188013]  [<ffffffff812fa8ee>] submit_bio+0x6e/0x140
[ 2720.188013]  [<ffffffffa0144063>] ext4_mpage_readpages+0x373/0x6a0 [ext4]
[ 2720.188013]  [<ffffffff811a1dd7>] __do_page_cache_readahead+0x177/0x200
[ 2720.188013]  [<ffffffff811a1fc8>] ondemand_readahead+0x168/0x2b0
[ 2720.188013]  [<ffffffff8119750d>] filemap_fault+0x2cd/0x420
[ 2720.188013]  [<ffffffffa0102f11>] ext4_filemap_fault+0x31/0x50 [ext4]
[ 2720.188013]  [<ffffffff811c01f7>] __do_fault+0x67/0xe0
[ 2720.188013]  [<ffffffff811c3a6e>] handle_pte_fault+0xbae/0x1660
[ 2720.188013]  [<ffffffff811c60da>] handle_mm_fault+0x2fa/0x640
[ 2720.188013]  [<ffffffff81067dda>] __do_page_fault+0x23a/0x4b0
[ 2720.188013]  [<ffffffff8106807b>] do_page_fault+0x2b/0x70
[ 2720.188013]  [<ffffffff8161f658>] page_fault+0x28/0x30
[ 2720.188013] DWARF2 unwinder stuck at page_fault+0x28/0x30
[ 2720.188013] 
[ 2720.188013] Leftover inexact backtrace:
[ 2720.188013] 
[ 2720.188013] Code: 48 b9 00 01 00 00 00 00 ad de 48 be 00 02 00 00 00 00 ad de 48 89 0a 48 89 72 08 8b 52 f0 85 d2 74 07 83 af 18 31 01 00 01 f3 c3 <0f> 0b 66 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 8b 87 94 00 00 
[ 2720.188013] RIP  [<ffffffffa00be685>] get_free_grant+0x55/0x60 [xen_blkfront]
[ 2720.188013]  RSP <ffff8800ffbc36d8>
[ 2720.188013] ---[ end trace 1b8113c29a7c42df ]---

Resolution

Workaround: Downgrade the existing kernel to version 4.4.143-94.47.1 

zypper in -f --oldpackage kernel-default-4.4.143-94.47.1

Additional Information

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:7023434
  • Creation Date: 11-Oct-2018
  • Modified Date:03-Mar-2020
    • SUSE Linux Enterprise Server

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.