SUSE Manager’s missing locking feature, and how it’s not missing at all

Earlier this month, a colleague from France asked why SUSE Manager doesn’t offer a system locking feature when you choose Salt as the client stack.

This feature is still available if you’re using the traditional SUSE Manager client stack. It allows you to lock the system and prevent any changes like installing or removing packages until the system is unlocked again.

If you can’t lock them in, what about a blackout instead?

At first, thought I’d just tell the colleague to file a feature request to bring this feature to Salt-based systems. But digging a bit deeper, I realized that Salt has a very similar feature built in: The Minion Blackout.

In minion blackout mode, the system will only react to a single command that allows switching the blackout mode off again. That is, unless you explicitly add more commands to a whitelist.

The cool thing is that activating and de-activating the blackout mode is as easy as setting a system parameter, or what Salt calls a Pillar.

Formulas with Forms to the rescue

I figured that this could be the perfect example for demonstrating the power of Formulas with Forms. If you aren’t familiar with Formulas with Forms yet, let me briefly explain what they are doing:

In Salt, you can describe how a system’s configuration should look like with so-called states. States use a simple YAML syntax do describe things like the software packages that should be installed on a system, services that should be running, or users that should exist.

When you combine several of those Salt states, put them into a folder and follow some basic rules and conventions, Salt calls them Formulas. The cool thing about SUSE Manager’s Formulas with Forms is that they expose the parameters users need to set to customize Formulas to a specific system in a nice web form. Or in Salt-speak: Formulas let you edit a system’s Pillar data.

We’ve been shipping a Formula for locale management for a while. And with Manager 3.2 we’ve added more of them, like a Formula for setting up a DNS server and one for managing DHCP. The web forms for working with such complex Formulas have become possible because of the massively improved Formulas with Forms framework in Manager 3.2.

Connecting the dots. Or: A new feature added in 5 minutes

Back to our original problem: The cool thing about the minion blackout feature is that we can actually implement it with a Form that has no Formula attached to it! Remember? It’s activated and de-activated by setting a Pillar to True or False.

So today I gave it a try and implemented a prototype of the “system lock” feature within 5 minutes, just using Formulas with Forms. 🙂

Step 1: The Formula

On my SUSE Manager test system, I created a directory called

blackout

in the

/srv/formula_metadata

directory. In that directory, I created a file called

form.yml

with the following contents:

minion_blackout:
  $name: Put server into blackout (no changes apply)
  $type: boolean

Step 2: There is no step 2!

Believe it or not: Those three lines of YML are really the only thing needed to implement the locking feature. Now my SUSE Manager has a blackout Formula listed in its Formula Catalog.

I can now add this Formula to one of my systems (or, via the System Groups feature, to a whole group of systems):

And that’s it: Now I have a nice UI to toggle blackout mode:

To prove that it’s actually working, activate the blackout mode (don’t forget to save the Form!) and schedule a restart for the system. Your system history should now show a failed action that gives you this error:

"ERROR executing \u0027system.reboot\u0027:
Minion in blackout mode.
Set \u0027minion_blackout\u0027 to False in pillar or grains
to resume operations.
Only saltutil.refresh_pillar allowed in blackout mode."

Uncheck the blackout mode, save again, and now your restart succeeds!

Well, actually, a mini-step 2

There is one more thing to do: As SUSE Manager will try to run the Salt states for the “blackout” Formula, there needs to be a folder

srv/salt/blackout

and in that folder an empty file

init.sls

that you can create with a

touch init.sls

from the command line. Otherwise you won’t be able to apply the Salt highstate (in other words, apply all configuration settings) for a system, regardless whether the blackout flag is set or not.

The usual disclaimers …

I haven’t tested whether this new feature has any side-effects. And while it does work out of the box with SUSE Manager’s re-boot feature, it may not prevent system changes triggered from the Salt command line until you manually sync the Pillars to the system under all circumstances. Although the Salt CLI seems to pick things up immediately:

This is Joachim Werner blogging live from the SUSE headquarters in Nuremberg, where the night has fallen and a lonesome Linux server is now in total blackout.