Error : "non-page-aligned MMIO BAR found" when trying to start XEN virtual machine

This document (7002873) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 11

Situation

XEN virtualization
When trying to start a virtual machine that has a PCI device directly bound to it using pciback, the following error shows up :
Error: pci:<pci-bus-ID> non-page-aligned MMIO BAR found
The domain fails to start.

This error occurs on SUSE Linux Enterprise Server (SLES) 11, which includes on XEN version 3.3.1.

This is caused by the fact that the memory range for device is not correctly aligned to a page limit. Starting with XEN 3.3/SLE 11 this is a requirement and devices that may have worked correctly on SLES 10 SP2 now cause this problem.

Resolution

The solution is to tell Domain0 (VM host server) kernel to remap the memory range for that specific PCI device towards the virtual machine. This is done by adding the following parameter to the kernel command line of Domain0 :
reassigndev=<pci-bus-ID>[,<pci-bus-ID>...]
An example : reassigndev=0000:02:00.0

This can be done from the YaST bootloader module by appending the parameter to the "Optional Kernel Command Line Parameter" string (not "Additional Xen Hypervisor Parameters").
After rebooting the host server, the virtual machine should start.

Additional Information

The following illustrates an example of the symptom, analysis and resolution of the problem :

t8:~ # xm start pcibacked-vm
Error: pci: 0000:02:00.0: non-page-aligned MMIO BAR found.

Examine the memory range(s) used by the device from the output of the hwinfo command. In this case it is a storage controller:
35: PCI 200.0: 0104 RAID bus controller
...
SysFS BusID: 0000:02:00.0
Hardware Class: storage
Model: "Silicon Image RAID bus controller"
...
Memory Range: 0xfeaffc00-0xfeaffc7f (rw,non-prefetchable,disabled)
Memory Range: 0xfeaf8000-0xfeafbfff (rw,non-prefetchable,disabled)
I/O Ports: 0xdc00-0xdc7f (rw,disabled)
Memory Range: 0xfea00000-0xfea7ffff (ro,prefetchable,disabled)
...

Since a memory page is 4096 bytes (hexadecimal 1000) the ranges should start with an address ending with three zeroes. One of the ranges above starts at 0xfeaffc00, which is not aligned to a full page address.

After implementing the solution the output of hwinfo has changed to :
...
35: PCI 200.0: 0104 RAID bus controller
...
SysFS BusID: 0000:02:00.0
Hardware Class: storage
...
Memory Range: 0xfea04000-0xfea0407f (rw,non-prefetchable)
Memory Range: 0xfea00000-0xfea03fff (rw,non-prefetchable)
I/O Ports: 0xdc00-0xdc7f (rw)
Memory Range: 0xf0000000-0xf007ffff (ro,prefetchable,disabled)
...
Now all memory ranges start at a full page address and the VM starts successfully.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.