After upgrading to SES 6 DeepSea stages fail with DNS / FQDN errors

This document (000019935) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Enterprise Storage 6 (SES 6)

Situation

The cluster was upgraded from SES 5.5 to SES 6, after this when running DeepSea stage 2, an error similar to the following is seen:
 
Exception: According to the 'domain' grain, the cluster does not have a DNS domain configured

When running stage 3, an error similar to the following is seen:
 
fqdn : ['fqdn deepsea_node.mydns.name does not match minion id deepsea_node']
domain : ["\nIn order to determine a `domain` which is required for some features in DeepSea you need to set-up the FQDN properly.\nExample:\nA device with the hostname `myhost` in the parent domain example.com has the fully qualified domain name myhost.example.com.

Resolution

Properly configure Salt and DeepSea to use FQDN's (Fully Qualified Domain Names).

Cause

With SES 5 using FQDN's were not required, but with SES 6 onwards this is a requirement.

Additional Information

To adjust Salt and DeepSea to use FQDN's take the following steps;

A. Salt:

1) Make a backup of everything under "/srv/pillar/ceph/proposals/".
2) Run `salt '*' pillar.items` and save the output somewhere for reference.
3) Change the hostname to FQDN on all nodes and verify after that `hostname -f` properly returns the expected FQDN.
4) Update the minion id in "/etc/salt/minion_id", to do this for all nodes at the same time, from the admin node run for example:
 
salt -I cluster:ceph cmd.run 'hostname -f > /etc/salt/minion_id'

5) Restart the salt-minion service on all nodes using `systemctl restart salt-minion.service`. This can be done for all minions by again from the admin node using for example:
 
salt -I cluster:ceph cmd.run 'systemctl restart salt-minion.service'

6) Rename the keys in "/etc/salt/pki/master/minions/" directory to match the new minion ids.

B. Adjust DeepSea:

1) Remove any existing '*.sls' files under "/srv/pillar/ceph/proposals", for example:
 
find /srv/pillar/ceph/proposals/ -name '*.sls' -exec rm '{}' ';'

2) Run DeepSea's stage 1 to regenerate all the SLS files with the correct minion IDs, before there should have been for example "/srv/pillar/ceph/proposals/role-mgr/cluster/ses-node01.sls" and other similarly named files. After there should be "/srv/pillar/ceph/proposals/role-mgr/cluster/ses-node01.mydns.name.sls" etc.
3) Edit "/srv/pillar/ceph/proposals/policy.cfg" and make sure the various lines are going to match the FQDN's correctly.
4) Run DeepSea stage 2 to update the pillar data.
5) Finally run `salt '*' pillar.items` again, and compare this output against the pillar data from step A. 2. The information should be the same, except for shortname->FQDN changes.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:000019935
  • Creation Date: 01-Apr-2021
  • Modified Date:01-Apr-2021
    • SUSE Enterprise Storage

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback@suse.com

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

Join Our Community

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.


SUSE Customer Support Quick Reference Guide SUSE Technical Support Handbook Update Advisories
Support FAQ

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.

Go to Customer Center