kiwi image build fails with KiwiDecodingError

This document (000019958) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 15 All versions

Situation

At the time of writing this document it in a specific locale setup is observed that kiwi build operations fail like in this scenario :
# kiwi-ng --debug --type docker system build --description . --allow-existing-root --target-dir /tmp/
[...]
[ DEBUG   ]: 13:17:42 | EXEC: [rpm --root /tmp/build/image-root -qa --qf %{NAME}|\n --changelog --dbpath /usr/lib/sysimage/rpm]
[ WARNING ]: 13:17:43 | Failed decoding literal. Forcing UTF-8 decoding
[ ERROR   ]: 13:17:43 | KiwiDecodingError: Locale setup is not utf-8 compatible


The problem is observed in a setup installed with en_US.UTF-8 locale :
# locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=


However the user has changed the LC_* settings to "C" :
# locale
LANG=en_US.UTF-8
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_PAPER="C"
LC_NAME="C"
LC_ADDRESS="C"
LC_TELEPHONE="C"
LC_MEASUREMENT="C"
LC_IDENTIFICATION="C"
LC_ALL=C

Resolution

In the shell invoking the kiwi-ng command set the environment variable :
LC_ALL=C.UTF-8

Cause

Setting LC_ALL to C results in LC_CTYPE seen by the externally run rpm command being C rather than C.UTF-8 and causes running rpm (to get --changelog of our packages) to generate output that includes extended ASCII characters that are not locale text and are not UTF-8. Which is the specific failure observed. To prevent this, the correct setting to use for LC_ALL is not C it is C.UTF-8. They are not equivalent in this context.

Is this expected behavior ? Yes and the cause has nothing to do with the kiwi configuration XML locale setting. rpm has no awareness of kiwi configuration and will only generate output supported by python (kiwi) if host configuration (locale environment) indicates to output UTF-8.

You can set everything except LC_ALL to C before running a python program and the LC_CTYPE seen by an external program to python that's run by python SubProcess will still be C.UTF-8. NB: That includes if you set LC_CTYPE itself to C before starting the python program then (as long as LC_ALL is empty) LC_CTYPE will still be C.UTF-8 when seen by the external program.

Setting LC_ALL to C.UTF-8 instead of C prevents the failure, but achieves the goal of the locale being C.
In other words, wanting to change the locale via environment variables that include LC_ALL when using python programs like kiwi should not be limited to LC_ALL=<locale name>. They should instead be expressed as the UTF-8 version of that locale, e.g. LC_ALL=C.UTF-8.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:000019958
  • Creation Date: 05-May-2021
  • Modified Date:05-May-2021
    • SUSE Linux Enterprise Server

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback@suse.com

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

Join Our Community

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.


SUSE Customer Support Quick Reference Guide SUSE Technical Support Handbook Update Advisories
Support FAQ

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.

Go to Customer Center