KVM live migration fails between SLES11 SP3 and SP4
This document (7017048) is provided subject to the disclaimer at the end of this document.
SUSE Linux Enterprise Server 11 Service Pack 4 (SLES 11 SP4)
Kernel-based Virtual Machine (KVM)
Error: "qemu: warning: error while loading state for instance 0x0 of device 'ram' load of migration failed"
When live migrating a KVM virtual machine, or restoring a saved image, the specifics of the new instance of the virtual machine must carefully match the old instance. This includes the memory layout of the guest.
Due to an unnoticed linker change, when building the SLE 11 SP4 kvm package, the size of the virtio-net pxe rom image (/usr/share/qemu-kvm/pxe.virtio.rom) changed from the expected <=64K size to > 64K, which impacts the runtime memory layout of the guest, preventing a clean migration or save/restore in some instances (see below).
SLE 11 SP1, SLE 12 GA, and SLE 12 SP1 x86 all have a consistent pxe-virtio.rom sizes, but the SLE 11 SP4 kvm package has been found to have included an incompatible sized pxe-virtio.rom file as described above. Unfortunately it would be extremely difficult to resolve this inconsistency for all supported migration configurations in a transparent way, due to the stringent requirements of the live migration and save/restore process.
The following are ways to avoid being impacted by this issue.
If at all possible do not do a live migration (or restore) of a guest using the virti-net interface from a KVM host with the pre kvm-1.4.2-35.1 package update to one with that update (or newer) applied. If you can off-line migrate your guests from the pre-patched KVM host to a patched KVM host, there will be no problem.
If your workflow is such that you require a live migration (or restore) from a pre-patched SLE 11 KVM host to a post-patched one (for example, due to an emergency migration need) you could temporarily replace the kvm-1.4.2-35.1 package /usr/share/qemu-kvm/pxe.virtio.rom file with the file from the old package (eg the one on the source host of the migration), and then restore the file as soon as the guest is migrated. You'll then want to shut down and restart the migrated guest as soon as reasonable to ensure it can continue to be migratable. Rebooting the guest is not sufficient, in this proposed workaround.
Note that if the pxe rom is not enabled for the virtio-net device, the issue will not occur. If no network booting through this interface is needed, this is also a way to avoid this issue. This is done at the libvirt level by specifying <rom bar='off'/> as an element for the virtio network device in the guest xml config. At the QEMU command line level, it is done by specifying romfile= or rombar=0 to the -device virtio-net-pci command line option.
The symptom of this incompatibility is that the live migration (or restore) will fail with the following message being produced by the QEMU executable:
qemu: warning: error while loading state for instance 0x0 of device 'ram' load of migration failed
This message is written to stderr by the QEMU program. When using libvirt to control KVM, it will be recorded in the per guest log file of on the KVM host of the target of the migration / restore. (eg: /var/log/libvirt/qemu/$vmname.log).
Note that it is possible for some other guest ram migration incompatibility not related to the above mentioned issue to generate this message, but it is quite unlikely.
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:7017048
- Creation Date: 04-Dec-2015
- Modified Date:03-Mar-2020
- SUSE Linux Enterprise Server
For questions or concerns with the SUSE Knowledgebase please contact: firstname.lastname@example.org