To take full advantage of VM Guest systems, it is sometimes necessary to assign specific PCI devices to a dedicated domain. When using fully virtualized guests, this functionality is only available if the chipset of the system supports this feature, and if it is activated from the BIOS.
This feature is available from both, AMD* and Intel*. For AMD machines, the feature is called IOMMU, in Intel speak, this is VT-d. Note that Intel-VT technology is not sufficient to use this feature for fully virtualized guests. To make sure that your computer supports this feature, ask your supplier specifically to deliver a system that supports PCI Pass-Through.
Some graphics drivers use highly optimized ways to access DMA. This is not always supported, and thus using graphics cards may be difficult.
When accessing PCI devices behind a PCIe bridge, all of the PCI devices must be assigned to a single guest. This limitations does not apply to PCIe devices.
Guests with dedicated PCI devices cannot be live migrated to a different host.
The configuration of PCI Pass-Through is twofold. First, the hypervisor must be informed that a PCI device should be available for reassigning. Second, the PCI device must be assigned to the VM Guest.
Select a device to reassign to a VM Guest. To do this run lspci and read the device number. For example, if lspci contains the following line:
06:01.0 Ethernet controller: Digital Equipment Corporation DECchip 21142/43 (rev 41)
In this case, the PCI number is 06:01.0.
Edit /etc/sysconfig/pciback, and add the PCI device number to the XEN_PCI_HIDE_LIST option, for example
As root, reload the pciback service:
Check if the device is in the list of assignable devices with the command
If you want to avoid restarting the host system, there is an alternative procedure to prepare the host system for PCI Pass-Through via the /sys/bus/pci file system:
Identify the PCI device and store it to a variable for easier handling.
# export PCI_DOMAIN_BUS_SLOT_FUNC=06:01.0
Check which driver is currently bound to the device and save its name to a variable.
# readlink /sys/bus/pci/devices/0000\:06:01.0/driver ../../../../bus/pci/drivers/igb # export DRIVER_NAME=igb
Detach the driver from the device, and load the pciback module.
# echo -n $PCI_DOMAIN_BUS_SLOT_FUNC > \ /sys/bus/pci/drivers/$DRIVER_NAME/unbind # modprobe pciback
Add a new slot to the pciback's list.
# echo -n $PCI_DOMAIN_BUS_SLOT_FUNC > \ /sys/bus/pci/drivers/pciback/new_slot
Bind the PCI device to pciback.
# echo -n $PCI_DOMAIN_BUS_SLOT_FUNC > \ /sys/bus/pci/drivers/pciback/bind
The device is now ready to be used in VM Guest by specifying 'pci=[$PCI_DOMAIN_BUS_SLOT_FUNC]' in the guest config file.
There are several possibilities to dedicate a PCI device to a VM Guest:
During installation, add the pci line to the configuration file:
HINT: If you want the Xen tools to manage preparing and assigning a PCI device to a VM Guest when it is activated, add managed=1 to the PCI setting in the guest configuration file, denoting that it is a 'managed' PCI device:
When the VM Guest is activated, the Xen tools will unbind the PCI device from its existing driver, bind it to pciback, and attach the device to the VM. When the VM is shut down, the tools will rebind the device to its original driver. When using the managed mode, there is no need to configure the hypervisor for PCI Pass-Through as described in Section 2.5.1, Configuring the Hypervisor for PCI Pass-Through.
The command xm may be used to add or remove PCI devices on the fly. To Add the device with number 06:01.0 to a guest with name sles11 use:
xm pci-attach sles11 06:01.0
To add the device to the Xend database, add the following section to the Xend database:
(device (pci (dev (slot 0x01) (domain 0x0) (bus 0x06) (vslt 0x0) (func 0x0) ) ) )
For more information about modifying the Xend database, see Section 5.3, Configuring a Virtual Machine by Modifying its Xend Settings.
After assigning the PCI device to the VM Guest, the guest system must care for the configuration and device drivers for this device.
Xen 4.0 and newer supports VGA graphics adapter pass-through on fully virtualized VM Guests. The guest can take full control of the graphics adapter with high performance full 3D and video acceleration.
VGA Pass-Through functionality is similar to PCI Pass-Through and as such also requires IOMMU (or Intel VT-d) support from the motherboard chipset and BIOS.
Only the primary graphics adapter (the one that is used when you power on the computer) can be used with VGA Pass-Through.
VGA Pass-Through is supported only for fully virtualized guests. Paravirtual guests (PV) are not supported.
The graphics card cannot be shared between multiple VM Guests using VGA Pass-Through — you can dedicate it to one guest only.
To enable VGA Pass-Through, add the following settings to your fully virtualized guest configuration file
where yy:zz.n is the PCI controller ID of the VGA graphics adapter as found with lspci -v on Domain0.
There are several resources that provide interesting information about PCI Pass-Through in the net: