chef-client INFO: HTTP Request Returned 409 Conflict: Client already exists

This document (7022917) is provided subject to the disclaimer at the end of this document.

Environment

SUSE OpenStack Cloud 6
SUSE OpenStack Cloud 7

Situation

Running chef-client errors out with:

root@d52-54-00-e1-0c-39:~ # chef-client
[2018-05-03T14:28:42+02:00] INFO: *** Chef 10.32.2 ***
[2018-05-03T14:28:42+02:00] INFO: Client key /etc/chef/client.pem is not present - registering
[2018-05-03T14:28:42+02:00] INFO: HTTP Request Returned 409 Conflict: Client already exists
[2018-05-03T14:28:42+02:00] INFO: HTTP Request Returned 403 Forbidden: You are not allowed to take this action.

================================================================================
Chef encountered an error attempting to create the client "d52-54-00-e1-0c-39.suse.com"
================================================================================

Authorization Error:
--------------------
Your validation client is not authorized to create the client for this node (HTTP 403).

Possible Causes:
----------------
* There may already be a client named "d52-54-00-e1-0c-39.suse.com"
* Your validation client (chef-validator) may have misconfigured authorization permissions.

[2018-05-03T14:28:42+02:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2018-05-03T14:28:42+02:00] FATAL: Net::HTTPServerException: 403 "Forbidden"

Resolution

On the admin node delete the client:

knife client delete d52-54-00-e1-0c-39.suse.com

Then on the failing node d52-54-00-e1-0c-39.suse.com run:

crowbar_join --setup

crowbar_join --start

This will delete the public key of the node from the chef-server. It won't change any of the node attributes.
The crowbar_join call will then generate a new key-pair.

Cause

File /etc/chef/client.pem is missing for some reason.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.