10.6 Adding a PCI Device with virsh

To dedicate and assign a PCI device to VM Guest with virsh, follow these steps:

  1. Identify the PCI device.

    Use the virsh nodedev-list or lspci -n commands to identify the PCI device designated for pass-through to VM Guest.

    The following command lists available PCI devices only:

    virsh nodedev-list | grep pci

    Note that PCI devices are identified by a string in the following format (8086 is a variable that represents Intel architecture, and **** stands for a four-digit hexadecimal code specific to each device):

    pci_8086_****

    Remember the PCI device number—you will need it in future steps.

  2. Gather the information about the domain, bus, and function:

    # virsh nodedev-dumpxml pci_8086_1d26
    <device>
      <name>pci_8086_1d26</name>
      <parent>computer</parent>
      <driver>
        <name>ehci_hcd</name>
      </driver>
      <capability type='pci'>
        <domain>0</domain>
        <bus>0</bus>
        <slot>29</slot>
        <function>0</function>
        <product id='0x1d26'>Patsburg USB2 Enhanced Host Controller #1</product>
        <vendor id='0x8086'>Intel Corporation</vendor>
        <capability type='virt_functions'>
        </capability>
      </capability>
    </device>
  3. Detach the device from the host system prior to attaching it to VM Guest.

    # virsh nodedev-detach pci_8086_1d26 
      Device pci_8086_1d26 detached
  4. Convert the bus, slot, and function value from decimal to hexadecimal, and prepend '0x' to tell the system that the value is hexadecimal. In our example, bus = 0, slot = 29, and function = 0. Their hexadecimal values are:

    # printf %x 0
    0
    # printf %x 29
    1d

    Bus and function hexadecimal numbers are '0x00', while slot number is '0x1d'.

  5. Run virsh edit on your domain, and add the following device entry in the <devices> section.

    <hostdev mode='subsystem' type='pci' managed='no'>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1d' function='0x00'/>
      </source>
    </hostdev>

    HINT: 'managed' vs. 'unmanaged'

    libvirt recognizes two modes for handling PCI devices: they can be either 'managed' or 'unmanaged'. In the managed case, libvirt will handle all the details of unbinding the device from the existing driver if needed, resetting the device, binding it to pci-stub before starting the domain, etc. When the domain is terminated or the device is removed from the domain, libvirt will unbind from pci-stub and rebind to the original driver in the case of a managed device. If the device is unmanaged, the user must take care to ensure all of these management aspects of the device are done before assigning it to a domain, and after the device is no longer used by the domain.

    In our example, the managed='no' option means that the device is 'unmanaged', and we need to take care of the related driver with the virsh nodedev-detach and virsh nodedev-attach commands. To switch the device mode to 'managed', replace the snippet with managed='yes', and skip the remaining steps (apart from starting the guest).

  6. Once the VM Guest system is ready to use the PCI device, tell the host to stop using it. First check what driver the host system is using for the PCI device.

    # readlink /sys/bus/pci/devices/0000\:00\:1d.0/driver
    ../../../bus/pci/drivers/pci-stub
  7. In our case, the pci-stub driver is loaded, so you can start the virtual machine. It will be able to use the PCI device automatically.

    # virsh start sles11

    HINT: When using a multi-function PCI device that does not support FLR (function level reset) or PM (power management) reset, you need to detach all its functions from the VM Host Server. The device must be reset for security reasons, and without FLR or PM reset, you must reset the whole device. libvirt will refuse to do this if a function of the device is still in use by the VM Host Server or another VM Guest.

    You can safely detach a device function from the VM Guest with the virsh nodedev-detach command.

    HINT: If your PCI device is not 'managed', and the driver controlling the PCI device is not pci-stub, you have to detach it from the device first:

    virsh nodedev-detach pci_8086_1d26

    HINT: If you are running SELinux on your host, you need to disable it for now with

    # setsebool -P virt_use_sysfs 1

    and then start the virtual machine.