The Art of Kernel Module Harmony

Every modern Linux system relies on a delicate runtime dance. The SUSE Linux Enterprise Server (SLES) kernel includes more than 2,000 loadable modules out of the box, with about 60 percent of them serving as hardware drivers. But what happens when you need to introduce a brand-new storage controller driver or swap out an existing module after release?

You enter the volatile world of the kernel Application Binary Interface (kABI). Kernel modules interact with the core operating system using exported symbols. To make sure everyone is speaking the exact same language, a version checksum (modversion) is added to each symbol. If a single data structure parameter or return type shifts recursively down the line, that checksum changes, and the module simply will not load.

While SUSE works hard to keep the kABI stable during minor maintenance and security updates, more intrusive updates in service packs are sometimes unavoidable. To survive these multi-year lifecycles without breaking your system, your external kernel modules must be packaged properly.

Welcome to the World of Kernel Module Packages

To make sure your drivers update smoothly alongside the kernel, SUSE uses standard structures called Kernel Module Packages, or KMPs. Developed in close collaboration with the Linux Foundation Driver Backport Workgroup, KMPs generate a main RPM package alongside specialized sub-packages for every supported kernel flavor using the %kernel_module_package macro.

When setting up your build environment, you need the right tools. Utilizing the %kernel_module_package_buildreqs macro inside your RPM spec file pulls essential components like kernel-source, kernel-syms, and kernel-devel straight into the build root.

Skipping the kernel-syms package might let your build finish, but it disables module symbol versions entirely. Your module will blindly look like a match for any kernel, leading to system malfunctions that are incredibly frustrating to diagnose. To prevent this and achieve a completely clean, reproducible environment, we highly recommend utilizing the build.rpm chroot script available in the SLES Package Hub extension.

The Secure Boot Resigning Dance

In an era where security must be guaranteed before the operating system even boots, the UEFI Secure Boot specification ensures that no unrecognized kernel module can load without an approved digital key. SUSE addresses this by allowing the EFI shim loader to accept keys approved directly by the system owner.

When you are packaging modules yourself, you can use OpenSSL to generate a private key and a valid X.509 certificate. By passing that certificate to the KMP macro, the build process automatically spins up an additional <name>-ueficert sub-package. This helper package handles the certificate installation and triggers the mokutil utility to enroll the public key.

But what if you are a developer without direct access to secure corporate signing infrastructures during the initial package build?

This is where the pesign-obs-integration package saves the day with its modsign-repackage utility. It lets you sign kernel modules inside an existing KMP by executing a distinct four-step workflow:

• Prepare the KMP files containing the kernel modules you need to resign.
• Resign the modules using modsign-repackage, which unpacks the RPM, signs the internal .ko files, and packs it back up alongside a matching <NAME>-ueficert companion RPM.
• Verify the modules using modsign-verify to check that the internal file signatures are legitimate.
• Resign the package itself. Because modsign-repackage strips away any existing package-level verification, you must re-sign the entire outer RPM shell so package managers can verify its integrity.

Remember, once a secure-boot-enabled KMP is installed, the system owner must reboot the machine to approve the newly enrolled key before the module can finally be loaded and used.

Patching the Installer with Agama

Sometimes you need a specific driver long before the target operating system is fully installed—such as a storage controller required just to see the root disk.

For these scenarios, you can use a Driver Update Disk (DUD) to patch or extend our modern Agama installer at boot time. Agama lets you host your DUD image or raw RPM packages on an external URI. By simply editing your boot loader options during media boot and appending inst.dud=URI_OF_FILE, Agama pulls the required modules directly into the installation environment.

Keep in mind that unlike SLES 15 environments, Agama won’t manually traverse local file systems looking for random DUD files. You must point the URI exactly to a single, specific file for Agama to process it seamlessly.

Streamlining Your Infrastructure

Properly aligning drivers with an ever-evolving kernel doesn’t have to be a guessing game. By wrapping your binaries inside standardized KMPs, relying on the robust kbuild infrastructure, and paying close attention to Secure Boot key enrollment, you ensure your external hardware drivers remain stable, secure, and perfectly synced.

For the complete story, head over to the SUSE Best Practices at our documentation portal and check out the latest Kernel Module Package Manual, authored by my colleague Scott Bahling, Head of Partner Technology Management.