Missing container metrics in Rancher monitoring for AKS clusters in Rancher v2.2 - v2.4
This document (000020029) is provided subject to the disclaimer at the end of this document.
Situation
Issue
There is a bug affecting Rancher monitoring v1 in Rancher v2.2 - v2.4, in which some metrics for Rancher-managed AKS clusters are missing. When viewing Grafana dashboards for affected clusters, some graphs will display a Not enough data for graph
or No Data
message.
Pre-requisites
- A Rancher v2.2 - v2.4 instance
- A Rancher-managed AKS cluster with monitoring enabled
Root cause
This is due to the Kubelet scrape port not being properly set for Prometheus, as the cluster is not being detected as an AKS instance.
Resolution
Simply adding the exporter-kubelets.https: true
answer in the advanced section of the monitoring configuration page is all that is needed.
Navigate to the cluster view in the Rancher UI for the relevant cluster and click Tools
> Monitoring
. Click Show advanced options
in the bottom-right corner, and add exporter-kubelets.https: true
under Answers
, per the screenshot below. Once you hit Save
container metrics should start being populated in Prometheus and will be visible in the Grafana dashboards.
It is worth noting that this is considered a "hidden answer" which means that it will not remain visible under Answers
on the monitoring configuration page. As a result, you will have to re-add this answer every time there is a monitoring update or configuration change.
Further reading
- This issue is tracked in GitHub issue #27550.
Disclaimer
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000020029
- Creation Date: 06-May-2021
- Modified Date:06-May-2021
-
- SUSE Rancher
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com