Jump to content
SUSE Linux Enterprise Server 12 SP4

Xen to KVM Migration Guide

SUSE Linux Enterprise Server 12 SP4

As the KVM virtualization solution is becoming more and more popular among server administrators, many of them need a path to migrate their existing Xen based environments to KVM. As of now, there are no mature tools to automatically convert Xen VMs to KVM. There is, however, a technical solution that helps convert Xen virtual machines to KVM. The following information and procedures help you to perform such a migration.

Publication Date: December 07, 2018
Important
Important: Migration Procedure Not Supported

The migration procedure described in this document is not fully supported by SUSE. We provide it as a guidance only.

1 Migration to KVM Using virt-v2v

This section contains information to help you import virtual machines from foreign hypervisors (such as Xen) to KVM managed by libvirt.

Tip
Tip: Microsoft Windows Guests

This section is focused on converting Linux guests. Converting Microsoft Windows guests using virt-v2v is the same as converting Linux guests, except in regards to handling the Virtual Machine Driver Pack (VMDP). Additional details on converting Windows guests with the VMDP can be found in the separate Virtual Machine Driver Pack documentation at https://www.suse.com/documentation/sle-vmdp-22/.

1.1 Introduction to virt-v2v

virt-v2v is a command line tool to convert VM Guests from a foreign hypervisor to run on KVM managed by libvirt. It enables paravirtualized virtio drivers in the converted virtual machine if possible. A list of supported operating systems and hypervisors follows:

Supported Guest Operating Systems
  • SUSE Linux Enterprise Server

  • openSUSE

  • RedHat Enterprise Linux

  • Fedora

  • Microsoft Windows Server 2003 and 2008

Supported Source Hypervisor
  • Xen

Supported Target Hypervisor
  • KVM (managed by libvirt)

1.2 Installing virt-v2v

The installation of virt-v2v is simple:

sudo zypper install virt-v2v

Remember that virt-v2v requires root privileges, so you need to run it either as root, or via sudo.

1.3 Preparing the Virtual Machine

Note
Note: Conditions for Skipping This Step

If running virt-v2v on SLES 12 SP1 or before, this step can be safely skipped. This step can also be ignored if the virtual machine is fully virtualized or if it runs on SLES 12 SP2 or later.

The Xen virtual machine must have default kernel installed. To ensure this, run zypper in kernel-default on the virtual machine.

1.4 Converting Virtual Machines to Run under KVM Managed by libvirt

virt-v2v converts virtual machines from the Xen hypervisor to run under KVM managed by libvirt. To learn more about libvirt and virsh, see Book “Virtualization Guide. Additionally, all virt-v2v command line options are explained in the virt-v2v manual page (man 1 virt-v2v).

Before converting a virtual machine, make sure to complete the following steps:

Procedure 1: Preparing the Environment for the Conversion
  1. Create a new local storage pool.

    virt-v2v copies the storage of the source virtual machine to a local storage pool managed by libvirt (the original disk image remains unchanged). You can create the pool either with Virtual Machine Manager, or virsh. For more information, see Book “Virtualization Guide”, Chapter 12 “Managing Storage”, Section 12.1 “Managing Storage with Virtual Machine Manager” and Book “Virtualization Guide”, Chapter 12 “Managing Storage”, Section 12.2 “Managing Storage with virsh.

  2. Prepare the local network interface.

    Check that the converted virtual machine can use a local network interface on the VM Host Server. It is usually a network bridge. If it is not defined yet, create it with YaST › System › Network Settings › Add › Bridge.

    Note
    Note: Mappings of Network Devices

    Network devices on the source Xen host can be mapped during the conversion process to corresponding network devices on the KVM target host. For example, the Xen bridge br0 can be mapped to the KVM network default. Sample mappings can be found in /etc/virt-v2v.conf. To enable these mappings, modify the XML rule as necessary and ensure the section is not commented out with <!-- and --> markers. For example:

     <network type='bridge' name='br0'>
       <network type='network' name='default'/>
     </network>
    Tip
    Tip: No Network Bridge

    If there is no network bridge available, Virtual Machine Manager can optionally create it.

virt-v2v has the following basic command syntax:

virt-v2v -i INPUT_METHOD -os STORAGE_POOL SOURCE_VM
input_method

There are two input methods: libvirt or libvirtxml. See the SOURCE_VM parameter for more information.

storage_pool

The storage pool you already prepared for the target virtual machine.

source_vm

The source virtual machine to convert. It depends on the INPUT_METHOD parameter: For libvirt, specify the name of a libvirt domain. For libvirtxml, specify the path to an XML file containing a libvirt domain specification.

Note
Note: Conversion Time

Conversion of a virtual machine takes a lot of system resources, mainly for copying the whole disk image for a virtual machine. Converting a single virtual machine typically takes up to 10 minutes, although virtual machines using very large disk images can take much longer.

1.4.1 Conversion Based on the libvirt XML Description File

This section describes how to convert a local Xen virtual machine using the libvirt XML configuration file. This method is suitable if the host is already running the KVM hypervisor. Make sure that the libvirt XML file of the source virtual machine, and the libvirt storage pool referenced from it are available on the local host.

  1. Obtain the libvirt XML description of the source virtual machine.

    Tip
    Tip: Obtaining the XML Files

    To obtain the libvirt XML files of the source virtual machine, you must run the host OS under the Xen kernel. If you already rebooted the host to the KVM-enabled environment, reboot back to the Xen kernel, dump the libvirt XML file, and then reboot back to the KVM environment.

    First identify the source virtual machine under virsh:

    root # virsh list
     Id    Name                           State
    ----------------------------------------------------
    [...]
      2     sles12_xen                     running
    [...]

    sles12_xen is the source virtual machine to convert. Now export its XML and save it to sles12_xen.xml:

    root # virsh dumpxml sles12_xen > sles12_xen.xml
  2. Verify all disk image paths are correct from the KVM host's perspective. This is not a problem when converting on one machine, but may require manual changes when converting using an XML dump from another host.

    <source file='/var/lib/libvirt/images/XenPool/SLES.qcow2'/>
    Tip
    Tip: Copying Images

    To avoid copying an image twice, manually copy the disk image(s) directly to the libvirt storage pool. Update the source file entries in the XML description file. The virt-v2v process will detect the existing disks and convert them in place.

  3. Run virt-v2v to convert to KVM virtual machine:

    root # virt-v2v sles12_xen.xml1 \
    -i LIBVIRTXML2 \
    -os remote_host.example.com:/exported_dir3 \
    --bridge br04 \
    -on sles12_kvm5

    1

    The XML description of the source Xen-based virtual machine.

    2

    virt-v2v will read the information about the source virtual machine form a libvirt XML file.

    3

    Storage pool where the target virtual machine disk image will be placed. In this example, the image will be placed on an NFS share /exported_dir on the remote_host.example.com server.

    4

    The target KVM-based virtual machine will use the network bridge br0 on the host.

    5

    The target virtual machine will be renamed to sles12_kvm to prevent name collision with the existing virtual machine of the same name.

1.4.2 Conversion Based on the libvirt Domain Name

This method is useful if you are still running libvirt under Xen, and plan to reboot to the KVM hypervisor later.

  1. Find the libvirt domain name of the virtual machine you want to convert.

    root # virsh list
     Id    Name                           State
    ----------------------------------------------------
    [...]
      2     sles12_xen                     running
    [...]

    sles12_xen is the source virtual machine to convert.

  2. Run virt-v2v to convert to KVM virtual machine:

    root # virt-v2v sles12_xen1 \
    -i libvirt2 \
    -os storage_pool3 \
    --network eth04 \
    -of qcow25 \
    -oa sparce6 \
    -on sles12_kvm

    1

    The domain name of the Xen-based virtual machine.

    2

    virt-v2v will read the information about the source virtual machine directly from the active libvirt connection.

    3

    The target disk image will be placed in a local libvirt storage pool.

    4

    All guest bridges (or networks) will be connected to a locally managed network.

    5

    Format for the disk image of the target virtual machine. Supported options are raw or qcow2.

    6

    If the converted guest disk space will be sparse or preallocated.

1.4.3 Converting a Remote Xen Virtual Machine

This method is useful if you need to convert a Xen virtual machine running on a remote host. As virt-v2v connects to the remote host via ssh, ensure that SSH service is running on the host.

Note
Note: Passwordless SSH Access

Starting with SLES 12 SP2, virt-v2v requires a passwordless SSH connection to the remote host. This means a connection using an SSH key added to the ssh-agent. See man ssh-keygen and man ssh-add for more details on this.

To connect to a remote libvirt connection, construct a valid connection URI relevant for your remote host. In the following example, the remote host name is remote_host.example.com, and the user name for the connection is root. The connection URI then looks as follows:

xen+ssh://root@remote_host.example.com/

For more information on libvirt connection URIs, see http://libvirt.org/uri.html.

  1. Find the libvirt domain name of the remote virtual machine you want to convert.

    root # virsh -c xen+ssh://root@remote_host.example.com/ list
     Id    Name                           State
    ----------------------------------------------------
      1     sles12_xen                     running
    [...]

    sles12_xen is the source virtual machine to convert.

  2. The virt-v2v command for the remote connection looks like this:

    root # virt-v2v sles12_xen \
    -i libvirt \
    -ic xen+ssh://root@remote_host.example.com/ \
    -os local_storage_pool \
    --bridge br0

1.5 Running Converted Virtual Machines

After virt-v2v completes successfully, a new libvirt domain will be created with the name specified with the -on option. If you did not specify -on, the same name as the source virtual machine will be used. The new guest can be managed with standard libvirt tools, such as virsh or Virtual Machine Manager.

Tip
Tip: Rebooting the Machine

If you completed the conversion under Xen as described in Section 1.4.2, “Conversion Based on the libvirt Domain Name”, you may need to reboot the host machine and boot with the non-Xen kernel.

2 Xen to KVM Manual Migration

2.1 General Outline

The preferred solution to manage virtual machines is based on libvirt; for more information, see http://libvirt.org/. It has several advantages over the manual way of defining and running virtual machines—libvirt is cross-platform, supports many hypervisors, has secure remote management, has virtual networking, and, most of all, provides a unified abstract layer to manage virtual machines. Therefore the main focus of this article is on the libvirt solution.

Generally, the Xen to KVM migration runs in the following basic steps:

  1. Make a backup copy of the original Xen VM Guest.

  2. OPTIONAL: Apply changes specific to paravirtualized guests.

  3. Obtain information about the original Xen VM Guest and update it to KVM equivalents.

  4. Shut down the guest on the Xen host, and run the new one under the KVM hypervisor.

Warning
Warning: No Live Migration

The Xen to KVM migration cannot be done live while the source VM Guest is running. Before running the new KVM-ready VM Guest, you are advised to shut down the original Xen VM Guest.

2.2 Back Up the Xen VM Guest

To back up your Xen VM Guest, follow these steps:

  1. Identify the relevant Xen guest you want to migrate, and remember its ID/name.

    # virsh list --all
    Id Name                 State
    ----------------------------------
     0 Domain-0             running
     1 SLES11SP3            running
    [...]
  2. Shut down the guest. You can do this either by shutting down the guest OS, or with virsh:

    # virsh shutdown SLES11SP3
  3. Backup its configuration to an XML file.

    # virsh dumpxml SLES11SP3 > sles11sp3.xml
  4. Backup its disk image file. Use the cp or rsync commands to create the backup copy. Remember that it is always a good idea to check the copy with the md5sum command.

  5. After the image file is backed up, you can start the guest again with

    # virsh start SLES11SP3

2.3 Changes Specific to Paravirtualized Guests

Apply the following changes if you are migrating a paravirtualized Xen guest. You can do it either on the running guest, or on the stopped guest using guestfs-tools.

Important
Important

After applying the changes described in this section, the image file related to the migrated VM Guest will not be usable under Xen anymore.

2.3.1 Install the Default Kernel

Warning
Warning: No Booting

After you installed the default kernel, do not try to boot the Xen guest with it, the system will not boot.

Before cloning the Xen guest disk image for use under the KVM hypervisor, make sure it is bootable without the Xen hypervisor. This is very important for paravirtualized Xen guests as they usually contain a special Xen kernel, and often do not have a complete GRUB 2 boot loader installed.

  1. For SLES 11 update the /etc/sysconfig/kernel file. Change the INITRD_MODULES parameter by removing all Xen drivers and replacing the with virtio drivers. Replace

    INITRD_MODULES="xenblk xennet"

    with

    INITRD_MODULES="virtio_blk virtio_pci virtio_net virtio_balloon"

    For SLES 12 search for xenblk xennet in /etc/dracut.conf.d/*.conf and replace them with virtio_blk virtio_pci virtio_net virtio_balloon

  2. Paravirtualized Xen guests are running a specific Xen kernel. To run the guest under KVM, you need to install the default kernel.

    Note
    Note: Default Kernel Is Already Installed

    You do not need to install the default kernel for a fully virtualized guests as it is already installed.

    Enter rpm -q kernel-default on the Xen guest to find out if the default kernel is installed. If not, install it with zypper in kernel-default.

    The kernel we are going to use to boot the guest under KVM must have virtio (paravirtualized) drivers available. Run the following command to find out. Do not forget to replace 4.12.14-94.37 with your kernel version:

    # find /lib/modules/4.12.14-94.37-default/kernel/drivers/ -name virtio*
    /lib/modules/4.12.14-94.37-default/kernel/drivers/net/virtio_net.ko
    /lib/modules/4.12.14-94.37-default/kernel/drivers/scsi/virtio_scsi.ko
    /lib/modules/4.12.14-94.37-default/kernel/drivers/block/virtio_blk.ko
    /lib/modules/4.12.14-94.37-default/kernel/drivers/char/virtio_console.ko
    /lib/modules/4.12.14-94.37-default/kernel/drivers/char/hw_random/virtio-rng.ko
    /lib/modules/4.12.14-94.37-default/kernel/drivers/virtio
    /lib/modules/4.12.14-94.37-default/kernel/drivers/virtio/virtio.ko
    /lib/modules/4.12.14-94.37-default/kernel/drivers/virtio/virtio_pci.ko
    /lib/modules/4.12.14-94.37-default/kernel/drivers/virtio/virtio_balloon.ko
    /lib/modules/4.12.14-94.37-default/kernel/drivers/virtio/virtio_ring.ko
    /lib/modules/4.12.14-94.37-default/kernel/drivers/virtio/virtio_mmio.ko
  3. Update /etc/fstab. Change any storage devices from xvda to vda.

  4. Update the boot loader configuration. Enter rpm -q grub2 on the Xen guest to find out if GRUB 2 is already installed. If not, install it with zypper in grub2.

    Now make the newly installed default kernel the default for booting the OS. Also remove/update the kernel command line options that may refer to Xen-specific devices. You can do it either with YaST (System › Boot Loader), or manually:

    • Find the preferred Linux boot menu entry by listing them all:

      cat /boot/grub2/grub.cfg | grep 'menuentry '

      Remember the order number (counted from zero) of the one you newly installed.

    • Set it the default boot menu entry:

      grub2-set-default N

      Replace N with the number of the boot menu entry you previously discovered.

    • Open /etc/default/grubfor editing, and look for GRUB_CMDLINE_LINUX_DEFAULT and GRUB_CMDLINE_LINUX_RECOVERY options. Remove/update any reference to Xen-specific devices. In the following example, you can replace

      root=/dev/xvda1 disk=/dev/xvda console=xvc

      with

      root=/dev/vda1 disk=/dev/vda

      Note that you need to remove all references to xvc-type consoles (such as xvc0).

  5. Update device.map in one of /boot/grub2 or /boot/grub2-efi directories. Change any storage device from xvda to vda.

  6. To import new default settings, run

    grub2-mkconfig -o /boot/grub2/grub.cfg

2.3.2 Update the Guest for Boot under KVM

  1. Update the system to use default serial console. List the configured consoles, and remove symbolic links to xvc? ones.

    # ls -l /etc/systemd/system/getty.target.wants/
    getty@tty1.service -> /usr/lib/systemd/system/getty@.service
    getty@xvc0.service -> /usr/lib/systemd/system/getty@xvc0.service
    getty@xvc1.service -> /usr/lib/systemd/system/getty@xvc1.service
    
    # rm /etc/systemd/system/getty.target.wants/getty@xvc?.service
  2. Update the /etc/securetty file. Replace xvc0 with ttyS0.

2.4 Update the Xen VM Guest Configuration

This section describes how to export the configuration of the original Xen VM Guest, and what particular changes to apply to it so it can be imported as a KVM guest into libvirt.

2.4.1 Export the Xen VM Guest Configuration

First export the configuration of the guest and save it to a file. A typical one may look like this:

# virsh dumpxml SLES11SP3
<domain type='xen'>
  <name>SLES11SP3</name>
  <uuid>fa9ea4d7-8f95-30c0-bce9-9e58ffcabeb2</uuid>
  <memory>524288</memory>
  <currentMemory>524288</currentMemory>
  <vcpu>1</vcpu>
  <bootloader>/usr/bin/pygrub</bootloader>
  <os>
    <type>linux</type>
  </os>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/lib/xen/bin/qemu-dm</emulator>
    <disk type='file' device='disk'>
      <driver name='file'/>
      <source file='/var/lib/libvirt/images/SLES_11_SP2_JeOS.x86_64-0.0.2_para.raw'/>
      <target dev='xvda' bus='xen'/>
    </disk>
    <interface type='bridge'>
      <mac address='00:16:3e:2d:91:c3'/>
      <source bridge='br0'/>
      <script path='vif-bridge'/>
    </interface>
    <console type='pty'>
      <target type='xen' port='0'/>
    </console>
    <input type='mouse' bus='xen'/>
    <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/>
  </devices>
</domain>

You can find detailed information on the libvirt XML format for VM Guest description at http://libvirt.org/formatdomain.html.

2.4.2 General Changes to the Guest Configuration

You need to make a few general changes to the exported Xen guest XML configuration to run it under the KVM hypervisor. The following applies to both fully virtualized and paravirtualized guests. Note that not all of the following XML elements need to be in your specific configuration.

Tip
Tip: Conventions Used

To refer to a node in the XML configuration file, an XPath syntax will be used throughout this document. For example, to refer to a <name> inside the <domain> tag

<domain>
  <name>sles11sp3</name>
</domain>

an XPath equivalent /domain/name will be used.

  1. Change the type attribute of the /domain element from xento kvm.

  2. Remove the /domain/bootloader element section.

  3. Remove the /domain/bootloader_args element section.

  4. Change the /domain/os/type element value from linux to hvm.

  5. Add <boot dev="hd"/> under the /domain/os element.

  6. Add the arch attribute to the /domain/os/type element. Acceptable values are arch=”x86_64” or arch=”i686”

  7. Change the /domain/devices/emulator element from /usr/lib/xen/bin/qemu-dm' to /usr/bin/qemu-kvm.

  8. For each disk associated with the paravirtualized (PV) guest, change the following:

    • Change the name attribute of the /domain/devices/disk/driver element from file to qemu, and add a type attribute for the disk type. For example, valid options include raw or qcow2.

    • Change the dev attribute of the /domain/devices/disk/target element from xvda to vda.

    • Change the bus attribute of the /domain/devices/disk/target element from xen to virtio.

  9. For each network interface card, do the following changes:

    • If there is model defined in /domain/devices/interface, change its type attribute value to virtio

      <model type=”virtio”>
    • Delete all /domain/devices/interface/script sections.

    • Delete all /domain/devices/interface/target elements if the dev attribute starts with vif or vnet or veth. If using a custom network then change the dev value to that target.

  10. Remove the /domain/devices/console element section if it exists.

  11. Remove the /domain/devices/serial element section if it exists.

  12. Change the bus attribute on the /domain/devices/input element from xen to ps2.

  13. Add the following element for memory ballooning features under the /domain/devices element.

    <memballoon model="virtio"/>
Tip
Tip: Device Name

<target dev='hda' bus='ide'/> controls the device under which the disk is exposed to the guest OS. The dev attribute indicates the "logical" device name. The actual device name specified is not guaranteed to map to the device name in the guest OS. Therefore you may need to change the disk mapping on the boot loader command line. For example, if the boot loader expects a root disk to be hda2 but KVM still sees it as sda2, change the boot loader command line from

[...] root=/dev/hda2 resume=/dev/hda1 [...]

to

[...] root=/dev/sda2 resume=/dev/sda1 [...]

In the case of paravirtualized xvda devices, change it to

[...] root=/dev/vda2 resume=/dev/vda1 [...]

Otherwise the VM Guest will refuse to boot in the KVM environment.

2.4.3 The Target KVM Guest Configuration

After having applied all the modifications mentioned above, you end up with the following configuration for your KVM guest:

<domain type='kvm'>
  <name>SLES11SP3</name>
  <uuid>fa9ea4d7-8f95-30c0-bce9-9e58ffcabeb2</uuid>
  <memory>524288</memory>
  <currentMemory>524288</currentMemory>
  <vcpu cpuset='0-3'>1</vcpu>
  <os>
    <type arch=”x86_64”>hvm</type>
    <boot dev="hd"/>
  </os>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type="raw"/>
      <source file='/var/lib/libvirt/images/SLES_11_SP2_JeOS.x86_64-0.0.2_para.raw'/>
      <target dev='vda' bus='virtio'/>
    </disk>
    <interface type='bridge'>
      <mac address='00:16:3e:2d:91:c3'/>
      <source bridge='br0'/>
    </interface>
    <input type='mouse' bus='usb'/>
    <graphics type='vnc' port='5900' autoport='yes' keymap='en-us'/>
    <memballoon model="virtio"/>
  </devices>
</domain>

Save the configuration to a file in your home directory. After you later import it, it will be copied to the default /etc/libvirt/qemu. Suppose you save the file as SLES11SP3.xml.

2.5 Migrate the VM Guest

After you updated the VM Guest configuration, and applied necessary changes to the guest OS, shut down the original Xen guest, and run its clone under the KVM hypervisor.

  1. Shut down the guest on the Xen host by running shutdown -h now as root from the console.

  2. Copy the disk files associated with the VM Guest if needed. A default configuration will require the Xen disk files to be copied from /var/lib/xen/images to /var/lib/kvm/images. The /var/lib/kvm/images directory may need to be created (as root) if you have not previously created a VM Guest.

  3. Create the new domain, and register it with libvirt:

    # virsh define SLES11SP3.xml
     Domain SLES11SP3 defined from SLES11SP3.xml
  4. Verify that the new guest is seen in the KVM configuration:

    virsh list –all
  5. After the domain is created, you can start it:

    # virsh start SLES11SP3
     Domain SLES11SP3 started

3 For More Information

For more information on libvirt, see http://libvirt.org.

You can find more details on libvirt XML format at http://libvirt.org/formatdomain.html.

For more information on virtualization with Xen and KVM, see the SUSE Linux Enterprise Server documentation at http://www.suse.com/doc/.

4 Documentation Updates

This section lists content changes for this document.

4.1 October 2016 (Release of SUSE Linux Enterprise Server 12 SP2)

5 Legal Notice

Copyright© 2006– 2018 SUSE LLC and contributors. All rights reserved.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled GNU Free Documentation License.

For SUSE trademarks, see http://www.suse.com/company/legal/. All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its affiliates. Asterisks (*) denote third-party trademarks.

All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its affiliates, the authors, nor the translators shall be held liable for possible errors or the consequences thereof.

Print this page