Search results: “secure boot”

Various security researchers and the grub2 team have published more security issues in grub2 today, which can be used to bypass the UEFI secure boot chain.
These security issues have the same scope as the BootHole issues from 2020. This attack requires root access to the bootloader used in Linux operating systems, GRUB2. It bypasses normal Secure Boot protections to persistently install malicious code which cannot be detected by the operating system.

Given the need for root access to the bootloader, the described attack appears to have limited relevance for most cloud computing, data center and personal device scenarios, unless these systems are already compromised by another known attack. However, it does create an exposure when untrusted users can access a machine, e.g. bad actors in classified computing scenarios or computers in public spaces operating in unattended kiosk mode. These are scenarios which Secure Boot was intended to protect against.

SUSE has released fixed grub2 packages which close the vulnerabilities for all SUSE Linux products, and is releasing corresponding Linux kernel packages, cloud image and installation media updates. Please follow the normal update procedure to install them. Should you be unsure about your company’s procedure, please consult your local system administrator.

To ensure that sophisticated attackers cannot reinstall old versions of grub2, software and hardware vendors are working together. Over time, vendors are going to update cryptographic keys in the BIOS for new computers, as well as to provide so-called DBX Exclusion List updates for existing computers. These can prevent unpatched systems and old installation media from starting. Please make sure you have installed all relevant bootloader and operating system updates for BootHole before installing a BIOS or DBX Exclusion List update to ensure continuity.

References:

• SUSE TID 000019892
• SUSE CVE-2020-25632
• SUSE CVE-2020-25647
• SUSE CVE-2020-27749
• SUSE CVE-2020-27779
• SUSE CVE-2020-14372
• SUSE CVE-2021-20225
• SUSE CVE-2021-20233

If you have any questions or concerns, please reach out to your SUSE contact. Security and reliability continue to be top priorities for SUSE because they are top priorities for our customers and partners. And as always, customers and partners come first.

1. Preboot Execution Environment (PXE)

We have to start with the standard PXE setup that is used for decades for disk-less client bootstrapping. The client requests an IP address and gets it together with the information where to find a boot file that can be loaded via TFTP. In our case the network connection is done via the EFI stack and the file the system is loading is an EFI application.

2. Copy relevant files to the right places

The relevant files that we need can simply be copied from the ISO image. First we need the the bootx64.efi file that is the shim that has an official digital signature from Microsoft. This file gets validated during Secure Boot and allows us to load and validate the grub.efi file that was signed with the “openSUSE Secure Boot CA”. Grub will then load its config file and offers the same menu we know from the ISO image.

The following steps are the same for openSUSE and SUSE Linux Enterprise:

#> mount -o loop openSUSE-Leap-15.0-DVD-x86_64.iso /mnt
#> cp /mnt/EFI/BOOT/{bootx64.efi,grub.efi,grub.cfg} /srv/tftpboot/
#> cp -r /mnt/boot /srv/tftpboot/

To run a full network installation we also need to provide the ISO image content. A simple way to do this is via HTTP. Just create an installation sub-directory with the full content of the ISO image. Best practice is to mount the ISO image inside the web server environment.

The 'Installation' menu-entry inside the grub.cfg file can then be extended with the netsetup= and install= parameters. Simply extend the linuxefi line with the following string:

netsetup=dhcp,all install=http://192.168.7.1/install/opensuse/leap150/

We can also customize the boot menu theme by editing the boot/x86_64/grub2-efi/themes/openSUSE/theme.txt file. In this example we will edit the theme title that is shown during boot.

title-text: "openSUSE Leap 15.0 (UEFI Network Installation)"

3. Showtime

You need to enable UEFI IPv4 Boot and Secure Boot inside the BIOS of your system. Then it should look similar to this KVM Secure Boot screen cast.

Have a lot of fun! 😉

In the previous posts, UEFI Secure Boot and Our Planned Approach to Secure Boot, Olaf Kirch has introduced you into the topic of UEFI Secure Boot and the basics of our approach to implementing it in SUSE. In this post, I’ll lead you through the technical details of our Secure Boot plan. So be prepared for this post to be rather detailed. And technical.

The goal of Secure Boot is to prevent malware from hiding embedded in the boot chain by performing a verification of every executed component starting with a fresh reboot of the whole platform. To achieve its goal, Secure Boot must prevent any modification of the verification process, the keys, or any other variables by untrusted code or untrusted entities. An example of an untrusted entity is a hacker who has penetrated the system through an unpatched security hole in the operating system.

There are two types of trusted users:

First, those who hold the keys. The Platform Key (PK) allows almost everything. The Key Exchange Key (KEK) allows all a PK can except changing the PK.

Second, anyone with physical access to the machine. A user with physical access can reboot the machine, and configure UEFI.

UEFI offers two types of variables to cater to the needs of those users:

The first is the so called “Authenticated Variables,” which can be updated from both within the boot process (the so called Boot Services Environment) and the running OS, but only when the new value of the variable is signed with the same key that the old value of the variable was signed. And they can either only be appended to or changed to a value with a higher serial number.

The second is the so called “Boot Services Only Variables.” These variables are accessible to any code that runs during the boot process. After the boot process ends and before the OS starts, the bootloader must call the ExitBootServices() call. After that, these variables are no longer accessible, the OS can’t touch them.

The various UEFI key lists are of the first type, as this allows on-line updating, adding and blacklisting of keys, drivers and firmware fingerprints. It’s the second type of variable, the “Boot Services Only Variable” that will help us in our quest for an implementation of Secure Boot that is both secure and open source friendly. And compatible with GPLv3.

As Olaf explained in the last post, we start with a shim, based on the Fedora shim, signed by either a certificate signed by the SUSE KEK or a Microsoft-issued certificate, based on what KEKs are available in the UEFI key database on the system.

This allows the shim to load and execute.

The shim then goes on to verify that the GRUB2 bootloader it wants to load is trusted. It will not use the SUSE KEK nor the Microsoft cert for this. In a default situation the shim will use an independent SUSE certificate embedded in its body. In addition, the shim will allow to “Enroll” additional keys, overriding the default SUSE key. Let’s call them “Machine Owner Keys” or MOKs for short.

The enrollment process begins by rebooting the machine and interrupting the boot process  (e.g., pressing a key) when the shim loads. The shim will then go into enrollment mode, allowing the user to replace the default SUSE key with keys from a file on the boot partition. If the user chooses to do so, the shim will then calculate a hash of that file and put the result in a “Boot Services Only” variable. This allows the shim to detect any change of the file made outside of Boot Services and thus avoid the tampering with the list of user approved MOKs.

An important aspect to remember is that all of this happens during boot time, only verified code is executing now. Therefore, only a user present at the console can say, “I want to use my own set of keys.” It can’t be malware or a hacker with remote access to the OS because hackers or malware can only change the file, but not the hash stored in the “Boot Services Only” variable.

GRUB2, once loaded and verified by the shim, will call back to the shim when it wants to verify the kernel – to avoid duplication of the verification code. The shim will use the same list of MOKs for this and tell GRUB2 whether it can load the kernel.

And that’s it.

This is all you need to be able to work on improving the kernel or the bootloader. Install a new set of keys and authorize them by being physically present during the first reboot.

Also, thanks to MOKs being a list and not just a single MOK, you can make the shim trust keys from several different vendors, allowing dual- and multi-boot from the GRUB2 bootloader.

In the end the real implementation may be a little bit more complicated – for example password-protecting the MOK authorization feature to allow secure authenticated updating of the MOK list from within the OS – but this is the gist of it. And since you can freely modify GRUB2 and your kernel as an owner of a machine, you are happy and so is  GPLv3 that the machine didn’t get tivoized.

You may be wondering whether this goes against the UEFI specification or Microsoft Win8 Logo requirements or any of the associated contracts.

I don’t think so – even the UEFI specification allows, but doesn’t mandate, that a UEFI implementation allow a user authorize EFI code with an invalid signature by adding a hash to a special variable.

And the Linux Foundation has asked vendors to make sure to include a way of clearing the PK to allow the user to enroll their own set of keys.

Our approach is just making sure that a feature like this is available everywhere.

To keep the “PC” a free platform.

In this follow-on blog to UEFI Secure Boot, I will describe our plans towards UEFI Secure Boot. Note that when we say “SUSE”, we really mean two very distinct distributions –  SUSE Linux Enterprise on one hand, and openSUSE on the other hand. The latter, being a  community project, is rather independent in their decisions on how to address the issue –  so the description below should be considered as the current Plan of Record for SUSE Linux Enterprise, but in the context of openSUSE, it can be a proposal for the community’s consideration only.

As explained in the previous installment of this series, UEFI Secure Boot is a useful technology, making it harder for attackers to hide a rootkit in the boot chain.

And at the same time, already the basics of its operation – establishing a single root of trust – conflict with the principles of Open Source development, which must be independent and distributed to work.

On top of that, there are a number of licensing and legal headaches waiting for anyone who wants to make use of Secure Boot in its default form. The GPL v3 anti-tivoization clause being one, another one being the wording of Microsoft’s SysDev contract that precludes the  signing of GPLv3 binaries with a Microsoft-provided certificate.

Currently, the first desktop systems are shipping with Secure Boot support, and we expect it to become pervasive in all newly sold PCs toward the end of this year. And of course we expect all of them to be shipped with Microsoft’s Key Exchange Key (KEK) installed by default.

Some of these new machines will probably have a way to disable secure boot easily; in others it may be possible but cumbersome; and in still others it may be impossible without loss of other functionality.

Supporting UEFI Secure Boot essentially boils down to having a boot loader with a digital signature that the firmware recognizes as a trusted key, and in order to be useful to Enterprise customers, that key should be trusted by the firmware a priori, without requiring any manual intervention.

There are two ways of getting there. One is to work with hardware vendors to have them endorse a SUSE key which we then sign the boot loader with. The other way is to go through Microsoft’s Windows Logo Certification program to have the boot loader certified and have Microsoft recognize our signing key (i.e. have it signed with their KEK). We are currently evaluating both approaches, and may eventually even pursue both in parallel.

At the implementation layer, we intend to use the shim loader originally developed by Fedora – it’s a smart solution which avoids several nasty legal issues, and simplifies the certification/signing step considerably. This shim loader’s job is to load grub2 and verify it; this version of grub2 in turn will load kernels signed by a SUSE key only. We are
currently considering to provide this functionality with SLE11 SP3 on fresh installations with UEFI Secure Boot present.

Now it is probably clear why this approach offers the level of security that UEFI Secure Boot promises – there is an unbroken chain of verification, making sure only trusted and signed code gets executed prior to handing control to the Operating System kernel.

What isn’t clear is how this allows Open Source developers to run their own kernels, or bootloaders for that matter or how this complies with the GPL v3 license. In the next part of this series of blogs, we will explain how we intend to provide this with our version of the shim.

In case you don’t know what this blog is talking about: UEFI is the “Unified Extensible Firmware Interface”, and “Secure Boot” is one of its more recent features that is generating a bit of a stir in the Open Source world.

At SUSE, we have been looking at UEFI Secure Boot long and hard.

On one hand, we agree that closing down some of the loopholes in the boot process is a worthwhile goal. For decades, we have accepted that this process has been one of the soft spots that are essentially unfixable without a major change in the BIOS. Now that this change is coming, we are ready to embrace it. And we’re also happy to see a much bigger change happening as part of this, which is the establishment of UEFI as the standard firmware on all x86 platforms.

On the other hand, as a Linux company, we are seeing a number of issues that have been causing us quite some headaches, which we wanted to resolve before we publicly state our plans in this regard.

In order to explain our concerns, let’s take a brief look at what Secure Boot really does, in a nutshell. In the world of UEFI, securing the bootstrapping process means establishing a chain of trust. The “platform” is the root of this chain of trust; I tend to think of it as the motherboard and the on-board firmware ROM. Or, put slightly differently, it’s the hardware vendor, and the chain of trust flows from that hardware vendor to the component manufacturers, the OS vendors, etc.

On the legal side, this trust is established by a lot of contracts and other legal paperwork. On the level of bits and bytes, this trust is expressed via public key cryptography.  The vendor puts a so-called Platform Key into the ROM, representing the root of trust. The trust relationships with operating system vendors and others is documented by signing their keys with the Platform Key.

Finally, security is established by requiring that no code will be executed by the firmware unless it has been signed by one of these “trusted” keys – be it an OS boot loader, some driver located in the ROM of some PCI Express card, or be it an update of the firmware itself.

Of course, one could make this a very long chain of trust, by requiring that everything that ever gets run on that machine be signed: the OS itself, the modules it loads, the binaries and shared libraries it loads, all the scripts executed by any random interpreter, and even
the commands you type into a shell session.

So obviously, that has to stop somewhere; and in the UEFI specification, that point is reached when the firmware hands control to the operating system. Essentially, if you want to use Secure Boot, you need to have your OS loader signed with a key trusted by the firmware, and you need the OS loader to verify that the kernel it loads can be trusted.

Admittedly, this is glossing over quite some details – but that is of no real importance for this discussion.

The crux of the matter is that this conflicts with the way the Open Source community has been developing code for several decades now.  The Open Source movement began as an act of emancipation from the operating system vendors of that time. Open Source was and is about Freedom, not in a dogmatic, but really in some very practical sense. The Linux community is where it is today because people could just start their own distribution, build their own boot loader and kernel, and grow a community. It is thriving the way it is thriving today because there are thousands of people around the globe who rebuild their kernel ten times a day to fix bugs, to add enhancements, or to run their test harness on the latest set of changes.  And it will only continue to thrive in this way if we keep it that way.

From this point of view, Secure Boot is very much at odds with the Linux development model.

Just for the record, I do not think this is a conspiracy, or a sinister attempt to kill Linux. That’s not going to happen. But while Secure Boot can be a significant improvement for many, it definitely creates obstacles for the Linux community.

Granted, the Windows 8 Logo certification requires that BIOS vendors should allow users to turn off Secure Boot on x86 platforms, and we have hope that all BIOS vendors will actually implement this in a way that works well for Linux developers. But ideally, Linux  developers should not be required to switch off a security feature in order to run their operating system.

Thus, over the past months, our guiding questions when dealing with Secure Boot have been, how can we make it work for our customers, and how can we reconcile the requirements of Secure Boot with the needs of the Linux community.

We plan to continue this series of blog postings soon with an overview of how we intend to support Secure Boot in SUSE Linux Enterprise, and what solutions we propose to the  openSUSE community.

A while back AWS introduced UEFI support for specific instance types. Then in March of 2023 AWS enabled hybrid-boot for AMIs and in addition to the memory encryption that has already been supported in EC2, AWS announced the support of attestation, also known as SEV-SNP recently. Throughout this process AWS and SUSE have been working closely together to ensure the support of these features in SUSE Linux Enterprise.

SUSE Linux Enterprise and openSUSE images have been ready for UEFI boot from day 1 and anyone was able to create an image that uses UEFI secure boot. However SLE and openSUSE images published by SUSE reamained set to boot with BIOS, until a few days ago. Any SLE image published with a date stamp of 20230428 or later is set to use the relatively new “uefi-preferred” boot mode setting when an image gets registered. openSUSE images with a datestamp of 20230504 and later have the same setting. This setting indicates that the image is capable of hybrid-boot, more on hybrid-boot below.

With this the logical question is why there has not been a UEFI bootable image published by SUSE until now?

The answer is fairly simply, duplication of images. Until March 6, 2023 an image for x86_64 was either set up for BIOS or UEFI booting. A hybrid boot setup was not recognized by the platform. With the enablement of hybrid boot in EC2 SLE  images with date stamps of 20230428 or later will now boot with UEFI secure boot or BIOS depending on the instance type. For openSUSE images this applies for images with a datestamp of 20230504 or later.

What does hybrid-boot mean?

In general it means that a system can boot either with EFI (Extendable Firmware Interface) of BIOS (Basic Inout/Output System) as fimware. In EC2 the “uefi-preferred” setting means that an x86_64 image will boot using UEFI Secure Boot for instance types that support UEFI and will boot using BIOS for instance types that do not supporte UEFI. Both firmware implementations are supported with the same image.

Support for attestation, i.e. SEV-SNP

Attestation requires that you use SLE 15 SP4 or later Service Packs or openSUSE Leap 15.4 or later images for your instances. Before you can follow the validation process that is part of the AWS documentation some extra steps are required.

1. Install necessary development packages
   1. zypper in git make gcc libopenssl-1_1-devel kernel-source linux-glibc-devel libuuid-devel automake autoconf gcc-c++
2. Update the headers only needed in instances launched with images with a date stamp less than v20230719
   1. cd /usr/src/linux-$VERSION
   2. make headers_install ARCH=x86_64 INSTALL_HDR_PATH=/usr/

After this you can follow the instructions in the AWS Documentation.

We are working on providing the sev-guest tools as a package and the steps above will eventually boil down to a simple zypper install command.

Today SUSE open-sources its implementation of a Secure Virtual Machine Service Module (SVSM), which will enhance the security of confidential virtual machines (CVMs). The COCONUT-SVSM will allow CVMs to securely store data in a trusted environment and emulate TPM devices on AMD EPYC Generation 3 and later. Written in the Rust programming language it features special isolation capabilities. It is dual-licensed under the Apache-2.0 and MIT licenses. With open-sourcing the project we invite the community to participate in its future development.

What is Confidential Computing

The very short answer is: Confidential Computing encrypts data during processing. In recent years CPU vendors have started to integrate features which allow to setup isolated and trusted execution environments that are inaccessible to the rest of the system. The SVSM builds on AMD Secure Virtual Machine with Secure Nested Paging (AMD SEV-SNP). With this extension the virtual machine runs with encrypted memory and register state, so that its content is not visible outside of the CVMs context.

Why a Secure VM Service Module

Confidential Computing is especially important in environments where the hypervisor is controlled by someone else and thus not fully trusted. Since all peripheral devices of a CVM are controlled by the hypervisor, these devices are not fully trusted too. For most devices like disks and network cards the CVM can protect itself from malformed or malicious input. But there are also devices, like the TPM, which contain security sensitive state that must not be visible to the hypervisor.

The SVSM allows to move the emulated TPM device from the hypervisor into the trusted CVM context. The TPM state is secure and the CVM can use it for attestation and storing encryption keys.

Getting and Installing the Code

The code for the COCONUT-SVSM is available on GitHub and split into several repositories:

• The SVSM source code is hosted at https://github.com/coconut-svsm/svsm.
  This repository contains an INSTALL.md file with detailed installation instructions.
• Linux kernel branches for svsm-host and svsm-guest support are at
  https://github.com/coconut-svsm/linux
• The QEMU svsm branch needed to run the SVSM is at
  https://github.com/coconut-svsm/qemu
• EDK2 firmware changes to support an SVSM are at
  https://github.com/coconut-svsm/edk2

Design Highlights

The SVSM code base has a rich set of features to build on. It contains around 8000 lines of Rust code and puts a strong emphasis on isolation. PerCPU page-tables isolate the mappings of data structures between the cores of a running system. The memory allocator uses buddy and slab algorithms and presents itself to the Rust runtime via the GlobalAlloc trait. With a global allocator the SVSM can use parts of the Rust library which rely on memory allocations, like vec and strings. But also other data structures and smart pointers like Box, Rc and Arc are available to use.

For robustness it includes a locking library with a spin-lock and read-write lock implementation. With fixups the SVSM can recover from exceptions that happen during request processing and return an error code to the operating system.

Last but not least there are also debugging features included, like the ability to collect and print the call-stack. The panic!() macro currently uses them to print a back-trace, but they can also help to debug locking issues.

Future of the COCONUT-SVSM

The released code base is far from being complete. It can boot a Linux image in a CVM with multiple vCPUs assigned. It does not include a TPM emulation yet, but we will add this as one of the very next steps. To implement this, the SVSM will get the ability to run separate modules at ring 3 within the SVSM context. This will isolate the modules from the core SVSM kernel and makes it easy to extend.

With a persistency layer modules like the TPM will be able to securely save and restore their state across reboots and shutdowns. Live migration support is also planned. The SVSM will be able to transfer the running guest operating system to another SVSM running on a remote machine. This requires a secure channel between the source and destination SVSM as well as help from the hypervisor for tracking dirty pages.

Ultimately we want to move more device emulation parts from the hypervisor into the CVM context to enable running almost unenlightened operating systems with AMD SEV-SNP protection.

Acknowledgments

The COCONUT-SVSM project would not have been possible without the close relationship to AMD. AMD provided the Linux kernel and OVMF modifications to complete the SVSM host and guest stack. Many thanks for the work and our continuous cooperation!

Security researchers from Eclypsium have published an attack called BootHole today. This attack requires root access to the bootloader used in Linux operating systems, GRUB2. It bypasses normal Secure Boot protections to persistently install malicious code which cannot be detected by the operating system.

Given the need for root access to the bootloader, the described attack appears to have limited relevance for most cloud computing, data center and personal device scenarios, unless these systems are already compromised by another known attack. However, it does create an exposure when untrusted users can access a machine, e.g. bad actors in classified computing scenarios or computers in public spaces operating in unattended kiosk mode. These are scenarios which Secure Boot was intended to protect against.

SUSE has released fixed grub2 packages which close the BootHole vulnerability for all SUSE Linux products, and is releasing corresponding Linux kernel packages, cloud image and installation media updates. Please follow the normal update procedure to install them. Should you be unsure about your company’s procedure, please consult your local system administrator.

To ensure that sophisticated attackers cannot reinstall old versions of grub2, software and hardware vendors are working together. Over time, vendors are going to update cryptographic keys in the BIOS for new computers, as well as to provide so-called DBX Exclusion List updates for existing computers. These can prevent unpatched systems and old installation media from starting. Please make sure you have installed all relevant bootloader and operating system updates for BootHole before installing a BIOS or DBX Exclusion List update to ensure continuity.

References:

• The SUSE TID
• Research paper “There is a hole in the boot” from Eclypsium
• CVE-2020-10713
• CVE-2020-14308
• CVE-2020-14309
• CVE-2020-14310
• CVE-2020-14311
• CVE-2020-15705
• CVE-2020-15707
• CVE-2019-20908
• CVE-2020-15780
• Microsoft ADV200011 Guidance for Addressing Security Feature Bypass in GRUB