Recommended update for python-kiwi

Announcement ID: SUSE-RU-2019:0734-1
Rating: moderate
References:
Affected Products:
  • SUSE Linux Enterprise Desktop 12 SP4
  • SUSE Linux Enterprise High Performance Computing 12 SP4
  • SUSE Linux Enterprise Server 12 SP4
  • SUSE Linux Enterprise Server for SAP Applications 12 SP4

An update that has eight fixes can now be installed.

Description:

This update for python-kiwi provides the following fixes: - Fix some code issues reported by new flake8 version.

  • Change the default value for bundler compression. If no compression is configured in the kiwi config file, the default was set to False. However this led to problems on the OBS side for images which have fixed storage disk sizes configured (for example Azure images which request 30G disk size per instance). Thus the default changed to True.

  • Fix grub theme lookup. If the theme was not found at the expected place an exception was thrown. However the alternative lookup code in /boot was not reached with that exception.

  • Add a runtime check for preferences metadata, specifically verifying that there is a packagemanager defined and an image version defined.

  • Support alternative EFI and grub modules paths. In SUSE products EFI binaries are historically located in /usr/lib*/efi. In a recent move to package grub2 as noarch, a collision between x86_64 and aarch64 has been identified, as both place platform-specific files to the same location. To fix this, a new location was devised: /usr/share/efi/$(uname -m). At the same time /usr/lib/grub2 will move to /usr/share/grub2. (fate#326960)

  • Fix Xen guest detection. Xen setup (e.g in the Amazon Cloud) is only supported for the x86_64 architecture. (bsc#1123186, bsc#1123185)

  • Fix the location of grub unicode font file. grub2 is expecting the unicode font under the fonts directory in the /boot/grub*/ depending on how the distribution installs grub2. (bsc#1119416)

  • Add container history metadata on umoci repack call. This change makes sure that umoci repack call includes history metadata and skips that in umoci config call.

  • Do not assume package manager is always there. This change modifies the behavior for zypper to not assume rpm binary is always part of the image. An image could be bootstrapped only without zypper or rpm, in that case it does not make sense and it is not possible to dump and reload the rpmdb.

  • Allow to switch off install image boot timeout. This commit adds a new attribute called: <type ... install_continue_on_timeout="true|false"/> It allows to setup the boot timeout for install images build with KIWI. If not set or set to 'true', the configured boottimeout or its default applies to the install image as it was before. If set to 'false' there will be no timeout in the install image bootloader setup and the boot only continues on manual intervention.

  • Make result compression in the bundler optional. Calling kiwi result bundle will take the image build results and bundle the relevant image files according to their image type. Depending on the result configuration this could instruct the bundler to compress one or more files from the result. If compression is activated the result image has to be uncompressed before it can be used.

  • Fix using SysConfig objects. Objects of that class do not provide a get method but overload the bracket [] operator. Using the get() method would fail.

  • Use chkstat to verify and fix file permissions. Call chkstat in system mode which reads /etc/sysconfig/security to determine the configured security level and applies the appropriate permission definitions from the /etc/permissions* files. It is possible to provide those files as overlay files in the image description to apply a certain permission setup when needed. Otherwise the default setup as provided on the package level applies. It is required that the image root system has chkstat installed. If not present KIWI will skip this step and continue with a warning.

  • Allow setting the protocol to tcp or upd (e.g. "80/tcp") for exposed container ports. If no protocol is provided, OCI defaults are applied.

  • Fix disk size calculation for VMX. Disk size calculation must take into account the empty volumes that are to be mounted in a directory that does not exist in the root tree, otherwise there is KeyError. The result of storate/setup._calculate_volume_mbytes must be a dict including all defined volumes.

  • More clarity on kernel version lookup. Lookup of the kernel version is done by directly reading the kernel image via a small tool named kversion. The scope of the tool is limited and does not work for e.g kernel images which contain their own decompressor code. For the special cases exceptions were defined, one was zImage. The recently added exception for vmlinuz seemed too intrusive and was also not well documented. This change tries to clarify and get back to explicit and easy to read coding.

  • Refactor kernel version lookup. Check the presence of the gzip compressed kernel binary and use it. If not present use the arbitrary kernel image format with the known limitations.

  • Refactor OCI tools. In order to provide buildah support some of the logic about temporary directories for OCI images creation needed to be moved to the dedicated OCI tool class. While umoci can operate in any directory and this is passed as an argument, this is not the case for buildah. In buildah workflow the storage path of work-in-progress images and containers and the mountpoint of the container rootfs are not customizable.

  • Use cow file on persistent grub live loop boot. When using tools like live-grub-stick, the live iso as generated by kiwi will be copied as file on the target device and a grub loopback setup is created there to boot the live system from file. In such a case the persistent write setup which tries to create an extra write partition on the target fails in almost all cases because the target has no free and unpartitioned space available. Therefore in case of such a loopback mounted system we create a cow file (live_system.cow) instead of a partition to setup persistent writing. The cow file will be created in the same directory the live iso image file was read from grub.

  • Better exception handling in OEM installer. If an error condition applies in the kiwi dump dracut code, the reaction was to stop the process with a dracut die() call. If the option 'rd.debug' was set on boot, this lead to a debugging shell which is good, but in a standard process this lead to a lock of the machine which is an unfortunate situation. This fix changes the behavior to always print the error message as a dialog message box on the primary console and reboot the system after keypress or timeout. In case of the debug switch configured the system die()'s as before.

  • Add parted dependency for dracut-kiwi-live package. dracut-kiwi-live requires the partprobe tool and this is provided by parted package. Persistent overlay setup fails if parted is not installed in the image.

  • Add support for --no-history umoci's flag. By using this flag kiwi appends only a single history entry for OCI containers.

  • Improve dialog usage in kiwi-dump-image. Dialog's "--radiolist" feature requires to navigate to the item, press "space" to select the item and then "enter" to execute. With "--menu", it is enough to just navigate to the item and press "enter" to execute, which is much more intuitive for most users.

  • Fixed OEM installer. In the implementation of the ramdisk installer, an error for the standard case was introduced such that the lsblk call was invalid. This led to no devices being present for the installation.

  • Fix rsync call for filesystem images. For filesystem images the rsync call was missing a trail slash for the source path causing the sync to include also the containing directory. With this change the filesystem image does not include the rootfs in any subdirectory.

  • Add history metadata for container builds. This change adds the history section in containerconfig. With that, 'author', 'created_by' and 'comment' can be customized. In addition, 'created' is always included with the image creation date time. 'created_by' entry is set to 'KIWI version' by default if nothing is provided.

  • Change bundling of image formats. By default none of the image formats were stored as compressed files. The reason behind this was the assumption that some formats automatically make use of compression, which is true but only in their processing and not in their data blocks at creation time. Storage and handling of the image file itself becomes cumbersome and therefore the default bundle setup for image formats was changed to be compressed. This means the image, as it gets packed by KIWI, needs to be uncompressed before use. The following image formats are affected by the change in a call of the result bundler:

  • qcow2 (.qcow2.xz)
  • vdi (.vdi.xz)
  • vhd (.vhd.xz)
  • vhdx (.vhdx.xz)
  • vmdk (.vmdk.xz)

  • Fixed firmware strip and lookup for kiwi initrd. In a kiwi initrd the function baseStripFirmware can be used to strip down the firmware to the actually used kernel drivers in that initrd. The code to do this was broken due to some other changes. This change fixes the method to work correctly again.

  • kiwi-partitions-lib: Wait for udev before lsblk. An LVM-enabled OEM image spuriously did not resize its PV / LVs due to lsblk sometimes racing with udev and the disk was just not available during get_partition_node_name(). Call udev_pending() before all lsblk calls to avoid that. (the lsblk man page also advocates this to synchronize with udev)

  • Refactor containerconfig xml evaluation. This change refactors the extracted data from containerconfig section to be tool agnostic.

  • Support ramdisk deployment in OEM images. Using the rd.kiwi.ramdisk boot option enables the deployment into a ramdisk. If this option is enabled, only ramdisk devices as provided by the brd kernel driver will be available for deployment.

  • Distinguish install and image dracut config. This fix distinguishes the files that should be installed inside the image dracut only than the ones installed in both, in install initrd and image initrd.

  • Apply OCI interface for container and root_import. Instead of directly calling the container archiving tool, in this case umoci, the code has been changed to use the new OCI interface class.

  • Added OCI tooling interface class. An initiative to formulate industry standards around container formats and runtime is available at https://www.opencontainers.org. Different tools to implement the specifications had been created. The purpose of this class and its sub-classes is to provide a common interface in kiwi to allow using all tools such that the container support in kiwi covers every linux distribution no matter what tooling was preferred.

  • Warn on modifications to intermediate configuration files. Some files are taken from the host and managed as intermediate config files during the build of the image. Changes to those files during the build run by e.g a script will not become effective because the file gets restored. With this fix the modification condition is detected and a warning message is displayed so that the author of the image can adapt the description as suggested in the message.

  • Move the default rpm database path into Defaults class.

  • Add a hardcoded rpm database path to import trusted keys so that they are in the expected location for zypper.

  • Allow simple path source in Uri class. This patch is needed as follow up fix for the setup of the package cache in local repositories. The is_remote method from the Uri class is used to identify if a repository source is remote or local. At that point the initial repository source was already translated into its components. In case of a local repository the Uri instance now receives a simple path and the is_remote method raised with a style error. This patch allows the Uri class to be more friendly and initializes a local path as file:/ typed source.

  • Do not cache packages from local repos for zypper. Access to packages from local repositories is as fast as reading them from a cache location. The additional package copy and cache update is superfluous and should be avoided.

  • Update /etc/machine-id management docs. Update the information about how /etc/machine-id is treated in KIWI and provide some hints for old systems where /var/lib/dbus/machine-id is not a symlink to /etc/machine-id.

  • Added machine id setup in dracut preparation. In case of a dracut booted image we empty out the systemd machine-id configuration file to trigger the rebuild of that information by the dracut boot code at boot time. This allows for unique systemd identifiers if the same image gets deployed on different machines. This also obsoletes the scripts people put in in config.sh or images.sh to solve this problem obsolete.

  • Add Codec utils for bytes literals decoding. In case of a literal decoding failure it tries to decode the result in utf-8. This is handy in python2 environments where python and the host might be using different charset configurations. In python3 this issue seems to be solved. (bsc#1110871)

  • Include livenet module with dmsquash-live support. The upstream dracut dmsquash-live module supports network mode with the livenet module. But that module must be explicitly included and is not fetched automatically.

  • Fixed URI handling with token query option. So far only the query format "?credentials=" was supported. In case of "?random_token_data" the returned uri was truncated and also the format check on the query caused a python trace. (bsc#1110869, bsc#1108508)

  • Make use of the quiet flag of mountpoint command. This sets the use of -q flag of mountpoint. Kiwi only checks the return code, thus any stdout is useless in this case.

  • Fixes LVM based image creation in OBS. Attempting to create LVM based images under the Open Build Service would run into some issues related to the fact that there is no udev running in the chroot environment used to build kiwi based images. Two workarounds have been implemented in this patch:

  • When calling lvcreate, include the -Zn option to disable the automatic zeroing of the header of the newly created LV device. Doing so requires that the LV device's /dev entry exists immediately after it has been created, but in a chroot environment udev is not going to be running to automatically populate /dev/<vg_name>/<lv_name> or /dev/mapper/<vg_name>-<lv_name>. This should be safe to do since the LV is being created within a loopback device based partition, which is backed by a zero filled file, created by qemu-img.
  • After creating an LV we need to run vgscan --mknodes to create the required device nodes under /dev, which won't be automatically created since udev is not running in the chroot environment.

  • Fix disk size calculation for VMX. Disk size calculation must take into account the empty volumes that are to be mounted in a directory that does not exist in the root tree otherwise there is KeyError. The result of storate/setup._calculate_volume_mbytes must be a dictionary including all defined volumes.

  • Fixed disk detection from root device. The method lookup_disk_device_from_root assigns the disk device matching the root device uuid. However in a multipath environment multiple disk devices matches the same root device. The code to assign the multipath map in this case was missing in the dracut code base. (bsc#1126283, bsc#1126318)

Patch Instructions:

To install this SUSE update use the SUSE recommended installation methods like YaST online_update or "zypper patch".
Alternatively you can run the command listed for your product:

  • SUSE Linux Enterprise Desktop 12 SP4
    zypper in -t patch SUSE-SLE-DESKTOP-12-SP4-2019-734=1
  • SUSE Linux Enterprise Server for SAP Applications 12 SP4
    zypper in -t patch SUSE-SLE-SAP-12-SP4-2019-734=1 SUSE-SLE-SERVER-12-SP4-2019-734=1
  • SUSE Linux Enterprise High Performance Computing 12 SP4
    zypper in -t patch SUSE-SLE-SERVER-12-SP4-2019-734=1
  • SUSE Linux Enterprise Server 12 SP4
    zypper in -t patch SUSE-SLE-SERVER-12-SP4-2019-734=1

Package List:

  • SUSE Linux Enterprise Desktop 12 SP4 (x86_64)
    • kiwi-tools-debuginfo-9.17.16-3.11.1
    • python-kiwi-debugsource-9.17.16-3.11.1
    • kiwi-tools-9.17.16-3.11.1
  • SUSE Linux Enterprise Server for SAP Applications 12 SP4 (x86_64)
    • kiwi-pxeboot-9.17.16-3.11.1
  • SUSE Linux Enterprise Server for SAP Applications 12 SP4 (ppc64le x86_64)
    • kiwi-man-pages-9.17.16-3.11.1
    • python2-kiwi-9.17.16-3.11.1
    • python-kiwi-debugsource-9.17.16-3.11.1
    • kiwi-tools-debuginfo-9.17.16-3.11.1
    • kiwi-tools-9.17.16-3.11.1
  • SUSE Linux Enterprise High Performance Computing 12 SP4 (aarch64 x86_64)
    • kiwi-man-pages-9.17.16-3.11.1
    • python2-kiwi-9.17.16-3.11.1
    • python-kiwi-debugsource-9.17.16-3.11.1
    • kiwi-tools-debuginfo-9.17.16-3.11.1
    • kiwi-tools-9.17.16-3.11.1
  • SUSE Linux Enterprise Server 12 SP4 (aarch64 ppc64le s390x x86_64)
    • kiwi-man-pages-9.17.16-3.11.1
    • python2-kiwi-9.17.16-3.11.1
    • python-kiwi-debugsource-9.17.16-3.11.1
    • kiwi-tools-debuginfo-9.17.16-3.11.1
    • kiwi-tools-9.17.16-3.11.1

References: