'SIGSEGV: segmentation violation' in prometheus container of the prometheus-project-monitoring-0 Pod when enabling Project monitoring on the System Project
This document (000020219) is provided subject to the disclaimer at the end of this document.
Enabling project monitoring in a Rancher v2.2 cluster, in which cluster monitoring is enabled, fails with the Prometheus Pod in a CrashLoopBackOff.
prometheus container in the
prometheus-project-monitoring StatefulSet fails with an error of the following format:
panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x669c12] goroutine 437 [running]: net/http.(*Client).deadline(0x0, 0xc005381070, 0x40bb8f, 0xc0055e3600) /usr/local/go/src/net/http/client.go:187 +0x22 net/http.(*Client).do(0x0, 0xc005cdaa00, 0x0, 0x0, 0x0) /usr/local/go/src/net/http/client.go:527 +0xab net/http.(*Client).Do(0x0, 0xc005cdaa00, 0x23, 0xc002802230, 0x9) /usr/local/go/src/net/http/client.go:509 +0x35 github.com/prometheus/prometheus/scrape.(*targetScraper).scrape(0xc0060fa960, 0x1fd4a60, 0xc00010ec60, 0x1fb2760, 0xc0002eb110, 0x0, 0x0, 0x0, 0x0) /app/scrape/scrape.go:471 +0x111 github.com/prometheus/prometheus/scrape.(*scrapeLoop).run(0xc00616a100, 0xdf8475800, 0x2540be400, 0x0) /app/scrape/scrape.go:813 +0x487 created by github.com/prometheus/prometheus/scrape.(*scrapePool).sync /app/scrape/scrape.go:336 +0x45d
- A cluster managed by Rancher v2.2
- Cluster monitoring enabled and Project monitoring enabled on the System project
Project monitoring is not compatible with the Rancher System project and should not be enabled in the System project. Starting with Rancher v2.3.0 monitoring of the System project is performed by cluster monitoring, when this is enabled, and the UI prevents enabling of project monitoring on the System project.
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000020219
- Creation Date: 06-May-2021
- Modified Date:06-May-2021
- SUSE Rancher
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com