Exploring Azure Monitor Datasource breaks via Grafana Ingress after upgrade of rancher-monitoring chart.
This document (000021830) is provided subject to the disclaimer at the end of this document.
Environment
Rancher version: 2.10.1
Rancher Monitoring chart : v105.1.1+up61.3.2
Situation
When attempting to access the Azure Monitor Datasource via the Grafana Ingress URL (by navigating to Data Sources > Explore for the configured Azure Monitor Data Source), the following error occurs:
grafana-azure-monitor-datasource plugin failed
Error: Error loading https://grafana.ingress.com/k8s/clusters/c-m-m2mshmdt/api/v1/namespaces/cattle-monitoring-system/services/http:rancher-monitoring-grafana:80/proxy/public/plugins/grafana-azure-monitor-datasource/module.js?_cache=11.1.0 (SystemJS Error#3 https://github.com/systemjs/systemjs/blob/main/docs/errors.md#3)
This issue arises after upgrading the Rancher Monitoring chart from version 104.1.2+up57.0.3 to 105.1.1+up61.3.2, or during a fresh installation of the Rancher Monitoring chart v105.1.1+up61.3.2 with Azure Monitors configured and when Grafana is accessed via its Ingress URL. The underlying cause is a change in how Grafana 11 loads JS files from plugins. While most changes were backward-compatible, in this scenario, Grafana incorrectly attempts to load the file from:
http://<grafana.ingress.domain>/api/v1/namespaces/cattle-monitoring-system/services/http:rancher-monitoring-grafana:80/proxy/public/plugins/grafana-azure-monitor-datasource/module.js?_cache=11.1.0
This loading fails due to URL rewriting issues.
Resolution
A permanent solution for this issue is tentatively scheduled for Rancher v2.11.2. Further details can be found here.
As a temporary workaround, you can update the Helm values for the Rancher Monitoring Chart by adding the following configuration to the grafana.ini
block:
grafana.ini:
server:
domain: <grafana-url>
protocol: http
root_url: '%(protocol)s://%(domain)s/grafana'
serve_from_sub_path: true
For example:
grafana.ini:
server:
domain: grafana.ingress.com
protocol: http
root_url: '%(protocol)s://%(domain)s/grafana'
serve_from_sub_path: true
Disclaimer
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000021830
- Creation Date: 07-May-2025
- Modified Date:21-May-2025
-
- SUSE Rancher
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com