SUSE Manager Proxy disappeared from SUSE Manager

This document (000020281) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Manager 4.1 Server
SUSE Manager 4.1 Proxy

Situation

A registered SUSE Manager Proxy disappeared from SUSE Manager Server. The proxy was not visible within the System Overview list in the WebUI but still visible within the Salt keys. When the salt-minion service was restarted on the proxy, the system appeared again in the SUSE Manager overview list, however, it lost its proxy role and it was visible as regular client/minion only. When the proxy was reconfigured using the proxy setup guide (configure-proxy.sh was used with the answer file used for the initial proxy setup), the proxy role was recovered, however, no clients were visible behind that proxy.

Resolution

There were 2 possible approaches how to restore clients:

remove them from SUSE Manager and bootstrap them again (which would mean loosing all its event history, formula configurations, group assignments etc.)
Re-activate them using re-activation key

Engineering provided a script "reactivate_proxy_client.py" which can be used to restore invisible clients behind the affected proxy. The script:

identifies which clients are supposed to be register via the affected proxy (based on proxy FQDN)
creates re-activation key for all affected clients
adds the re-activation key into "/etc/salt/minion.d/susemanager.conf"
restarts salt-minion service on affected clients

All clients were properly visible behind Proxy once they were re-activated using the mentioned script.
The script was made available as a pull request to the uyuni-project.

Cause

It is not clear why the Proxy vanished from SUSE Manager. There is a chance that it was removed by accident, however, there are no logs which would prove that. During the night when the issue happened, there was a time shift on the affected Proxy - time was moved back to Jan-19-2021. However, there is no data which would confirm that this caused the issue.

Additional Information

Interesting is that the clients were still communicating via the Proxy, they were just not visible/shown behind the Proxy in the WebUI. It was tested using the following approach:

1. Run the following command on SUSE Manager:
salt client_name grains.item master (on SUSE Manager)

2. Stop the salt-broker on SUSE Manager Proxy:

systemctl stop salt-broker

3. run the salt command from 1. again, it will hang

4. As soon the salt-broker on the SUSE Manager Proxy is restarted (systemctl start salt-broker), a reply from salt was received immediately.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.