2.5 PCI Pass-Through

To take full advantage of VM Guest systems, it is sometimes necessary to assign specific PCI devices to a dedicated domain. When using fully virtualized guests, this functionality is only available if the chipset of the system supports this feature, and if it is activated from the BIOS.

This feature is available from both, AMD* and Intel*. For AMD machines, the feature is called IOMMU, in Intel speak, this is VT-d. Note that Intel-VT technology is not sufficient to use this feature for fully virtualized guests. To make sure that your computer supports this feature, ask your supplier specifically to deliver a system that supports PCI Pass-Through.

Limitations

The configuration of PCI Pass-Through is twofold. First, the hypervisor must be informed at boot time that a PCI device should be available for reassigning. Second, the PCI device must be assigned to the VM Guest.

2.5.1 Configuring the Hypervisor for PCI Pass-Through

  1. Select a device to reassign to a VM Guest. To do this run lspci and read the device number. For example, if lspci contains the following line:

    06:01.0 Ethernet controller: Digital Equipment Corporation DECchip 21142/43 (rev 41)

    In this case, the PCI number is (06:01.0).

  2. Run YaST > System > Boot Loader.

  3. Select the Xen section and press Edit.

  4. Add the PCI number to the Optional Kernel Command Line Parameter line:

    pciback.hide=(06:01.0)
  5. Press OK and finish YaST.

  6. Reboot the system.

  7. Check if the device is in the list of assignable devices with the command

    xm pci-list-assignable-devices

Solution without Host System Restart

If you want to avoid restarting the host system, there is an alternative procedure to prepare the host system for PCI Pass-Through via the /sys/bus/pci file system:

  1. Identify the PCI device and store it to a variable for easier handling.

    # export PCI_DOMAIN_BUS_SLOT_FUNC=06:01.0
  2. Check which driver is currently bound to the device and save its name to a variable.

    # readlink /sys/bus/pci/devices/0000\:06:01.0/driver
    ../../../../bus/pci/drivers/igb
    # export DRIVER_NAME=igb
  3. Detach the driver from the device, and load the pciback module.

    # echo -n $PCI_DOMAIN_BUS_SLOT_FUNC > \
    /sys/bus/pci/drivers/$DRIVER_NAME/unbind
    # modprobe pciback
  4. Add a new slot to the pciback's list.

    # echo -n $PCI_DOMAIN_BUS_SLOT_FUNC > \
    /sys/bus/pci/drivers/pciback/new_slot
  5. Bind the PCI device to pciback.

    # echo -n $PCI_DOMAIN_BUS_SLOT_FUNC > \
    /sys/bus/pci/drivers/pciback/bind

The device is now ready to be used in VM Guest by specifying 'pci=[$PCI_DOMAIN_BUS_SLOT_FUNC]' in the guest config file.

2.5.2 Assigning PCI Devices to VM Guest Systems

There are several possibilities to dedicate a PCI device to a VM Guest:

Adding the device while installing:

During installation, add the pci line to the configuration file:

pci=['06:01.0']

HINT: If you want the Xen tools to manage preparing and assigning a PCI device to a VM Guest when it is activated, add managed=1 to the PCI setting in the guest configuration file, denoting that it is a 'managed' PCI device:

pci=['06:01.0,managed=1']

When the VM Guest is activated, the Xen tools will unbind the PCI device from its existing driver, bind it to pciback, and attach the device to the VM. When the VM is shut down, the tools will rebind the device to its original driver. When using the managed mode, there is no need to configure the hypervisor for PCI Pass-Through as described in Section 2.5.1, Configuring the Hypervisor for PCI Pass-Through.

Hot adding PCI devices to VM Guest systems

The command xm may be used to add or remove PCI devices on the fly. To Add the device with number 06:01.0 to a guest with name sles11 use:

xm pci-attach sles11 06:01.0
Adding the PCI device to Xend

To add the device to the Xend database, add the following section to the Xend database:

(device
    (pci
        (dev
            (slot 0x01)
            (domain 0x0)
            (bus 0x06)
            (vslt 0x0)
            (func 0x0)
        )
    )
)
      

For more information about modifying the Xend database, see Section 5.3, Configuring a Virtual Machine by Modifying its Xend Settings.

After assigning the PCI device to the VM Guest, the guest system must care for the configuration and device drivers for this device.

2.5.3 VGA Pass-Through

Xen 4.0 and newer supports VGA graphics adapter pass-through on fully virtualized VM Guests. The guest can take full control of the graphics adapter with high performance full 3D and video acceleration.

Limitations

  • VGA Pass-Through functionality is similar to PCI Pass-Through and as such also requires IOMMU (or Intel VT-d) support from the motherboard chipset and BIOS.

  • Only the primary graphics adapter (the one that is used when you power on the computer) can be used with VGA Pass-Through.

  • VGA Pass-Through is supported only for fully virtualized guests. Paravirtual guests (PV) are not supported.

  • The graphics card cannot be shared between multiple VM Guests using VGA Pass-Through — you can dedicate it to one guest only.

To enable VGA Pass-Through, add the following settings to your fully virtualized guest configuration file

gfx_passthru=1 
pci=['yy:zz.n']

where yy:zz.n is the PCI controller ID of the VGA graphics adapter as found with lspci -v on Domain0.

2.5.4 For More Information

There are several resources that provide interesting information about PCI Pass-Through in the net: