Converting deprecated libcgroup configurations

This document (7023842) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 12 Service Pack 4 (SLES 12 SP4)

Situation

In SLES 12 SP3 libcgroup has been deprecated. In SLES 12 SP4 the libcgroup1 and libgroup-tools packages have been removed. This TID gives assistance for moving to a non-libcgroup setup.3.1) Using controllers not handled by systemd

Resolution

1) What controllers are used (cgconfig.conf)?

## cpuset
Systemd does not handle cpuset controller.

Similar effect on scheduling can be achieved by calling sched_setaffinity(2) and set_mempolicy(2) directly from a restricted program.

systemd provides CPUAffinity= directive that uses sched_setaffinity(2), there is no counterpart for set_mempolicy(2) though.

If the suggestions above does not cover the use case, see 3.1) below.

## cpu
CPUShares=, CPUQuota=
(see `man systemd.resource-control`)

NOTE: Realtime group scheduling attributes of the controller are not supported by systemd.

## cpuacct
CPUAccounting=
(see `man systemd.resource-control`)

## memory
MemoryLimit=
(see `man systemd.resource-control`)

## devices
DevicePolicy=, DeviceAllow=

(see `man systemd.resource-control`)
## freezer
Manual use.

## net_cls
NetClass=
(see `man systemd.resource-control`)

## blkioBlockIOWeight=, BlockIODeviceWeight=, BlockIOReadBandwidth=
(see `man systemd.resource-control`)

2) What are the hierarchies (cgconfig.conf)?

If the controller is handled by systemd, the hierarchy tree is built from slice units (inner nodes) and service or scope units (leaves).

If the controller is not handled by systemd, the hierarchy must be created and configured manually.

Example:

mkdir -p /sys/fs/cgroup/$CONTROLLER/path/to/cgroup
echo $VALUE >/sys/fs/cgroup/$CONTROLLER/path/to/cgroup/$ATTRIBUTE

There is an unsupported helper Python script that parses configuration in libcgroup format and creates the hierarchy with attributes set. It can be requested from support as needed.

3) What processes are classified (cgrules.conf)?

NOTE: Asynchronous classification as performed by cgrulesengd is unreliable and proper way to classify processes is to fork(2) a new one, attach it to a cgroup and then exec(2) the to-be-classified program.

If the process to classify is a daemon, then configure proper directives of a respective systemd service (`man systemd.resource-control`).

If the process to classify is a one-time task, then a systemd-run can be used to realize the fork(2)-classify-exec(2) idiom.

Examples:

systemd-run --scope --slice=restricted.slice $PROGRAM
systemd-run --scope -p MemoryLimit=256M $PROGRAM

NOTE: systemd-run can only be run by privileged user.

3.1) Using controllers not handled by systemd

This applies to cpuset particularly and relies on cset utility. It can be used as a wrapper and can be composed with other commands (when run as privileged
user):

systemd-run --scope --slice=restricted.slice cset proc --exec $CGROUP $PROGRAM

[Service]
Slice=restricted.slice
ExecStart=/usr/bin/cset proc --exec $CGROUP $DAEMON

3.1.1) Classifying processes of unprivileged users

The permissions in the particular controller hierarchy allow only privileged users to modify it. Hence the process must be classified before dropping privileges.

cset proc --exec $CGROUP \
  systemd-run --scope --uid=$T_UID --slice=restricted.slice $PROGRAM
        
systemd-run --scope --slice=restricted.slice \
  cset proc -u $T_UID --exec $CGROUP $PROGRAM

[Service]
Slice=restricted.slice
ExecStart=/usr/bin/cset proc -u $T_UID --exec $CGROUP $PROGRAM

NOTE: It is not possible User= directive of service files because UID change would happen before cset is executed.

3.1.2) Implicit classification

Implicit classification is yielded by wildcard rule at the end of cgrules.conf

    *    cpuset    target/cgroup

Such a rule typically serves to divert all system tasks away from root cpuset on partitioned system. For that please see the chapter about shielding using cset (https://www.suse.com/documentation/slerte-12/book_slert_shielding/data/cha_shielding_model.html).

NOTE: Similar behavior can be achieved with isolcpus= kernel command line. This has two major drawbacks: a) any process can do sched_setaffinity(2) to an isolated CPU, b) there is no load balancing among isolated CPUs. Hence it is discouraged.

Cause

Additional Information

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.