Rancher pre v1.6.22 "Hosts stuck Reconnecting" Rancher server logs show 'Cursor returned more than one result'
This document (000020182) is provided subject to the disclaimer at the end of this document.
Hosts getting stuck in either state Reconnecting or Finishing-Reconnect and Rancher server logs include errors like the following:
2019-02-26 12:05:55,265 ERROR [51e1303d-21b2-409f-ba2e-7542e8de4941:9663402] [healthcheckInstanceHostMap:445975] [healthcheckinstancehostmap.remove]  [ecutorService-3] [c.p.e.p.i.DefaultProcessInstanceImpl] Unknown exception org.jooq.exception.InvalidResultException: Cursor returned more than one result
Rancher version lower than 1.6.22
In the Rancher MySQL database, find all the duplicates by checking column 3 for entries with more than a count of 1 in the return from the following query:
select host_id,healthcheck_instance_id,count(*) from healthcheck_instance_host_map where removed is null group by host_id,healthcheck_instance_id order by 3;
For each healthcheck_instance_id in any row with more than 1 in column 3, run the following command:
update healthcheck_instance_host_map set state='removed', removed=now(), remove_time=now() where healthcheck_instance_id='<INSERT_HEALTHCHECK_ID>';
Wait and watch the hosts view. The hosts should all finish reconnecting and instances should update.
Upgrade to 1.6.22+ or 2.x
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000020182
- Creation Date: 13-Jul-2021
- Modified Date:13-Jul-2021
- SUSE Rancher
For questions or concerns with the SUSE Knowledgebase please contact: firstname.lastname@example.org