Interrupts not distributed as specified in smp_affinity
This document (7018478) is provided subject to the disclaimer at the end of this document.
SUSE Linux Enterprise Server 12 Service Pack 1 (SLES 12 SP1)
SUSE Linux Enterprise Server 12
By manipulating the bitmask it is possible to change the set of permitted CPUs.
This is briefly documented in /usr/src/linux/Documentation/IRQ-affinity.txt.
For more details see the "Additional information" section.
However the outcome is not always as expected.
As an example it has been observed that when defining a group of CPUs, only the least bit CPU increments interrupts. In other words if for example CPUs 4-7 are masked (hex f0), then only CPU 4 is being used.
To establish whether the kernel has switched APIC model or it is operating in flat mode, grep in the kernel log.
# dmesg | grep -i apiccan (amongst other) show information like
[...] x2apic enabled
[...] Switched APIC routing to cluster x2apic.
[...] Switched APIC routing to physical flat.
The catch is that if nothing is mentioned about switching APIC routing, then flat mode is used.
By default QEMU/KVM enables x2apic for virtual machines.
Make sure to have a maximum of 8 CPUs available and that the "flat" mode is used as per above.
For systems enabling x2apic, add the "nox2apic" option to the kernel command line in the bootloader configuration.
It seems not possible to make a system, where the kernel switches to "physical flat", run in "flat" mode.
Furthermore it depends on the APIC (Advanced Programmable Interrupt Controller) model chosen by the Linux kernel during boot.
This model depends on the actual hardware APIC present in the system.
Examples of common APIC models are flat, physical flat, physical x2apic, cluster x2apic.
By default Linux is going to use "flat" APIC model, which is the only one enabling interrupt distribution across multiple CPUs.
During early probing in the boot process the kernel decides if "flat" is the correct APIC model to be used with the system. If not, it switches the model.
Looking at the rightmost hexadecimal digit, the following "table" illustrates the representations of the bitmasks.
CPU Binary Hex
CPU 0 0001 1
CPU 1 0010 2
CPU 2 0100 4
CPU 3 1000 8
By combining these bit patterns (simply adding the hex values) multiple CPUs can be defined.
For example a hex value of 02 translates into CPU 1, 20 equals CPU 5 and 0f means CPU 0-3.
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:7018478
- Creation Date: 11-Jan-2017
- Modified Date:03-Mar-2020
- SUSE Linux Enterprise Server
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com