How to create a custom cluster role in Rancher v2.x to grant permission on the metrics endpoint of the kube-apiserver in a Rancher managed cluster
This document (000020052) is provided subject to the disclaimer at the end of this document.
Situation
Task
This article details how to create a cluster role to grant users access to the /metrics
endpoint of the Kubernetes API Server, in Rancher-managed Kubernetes clusters.
Pre-requisites
- A Rancher v2.x instance
- A Rancher-managed Kubernetes cluster, with Metrics Server deployed. This is deployed by default in Rancher-provisioned clusters.
Resolution
In Rancher v2.4+ it should be possible to define a non-resource URL grant via role creation within the Rancher UI. However, this is affected by the issue tracked in Issue #30321 and use of the Rancher v3 API is therefore required to create the role.
-
The first step is to create a custom cluster role within Rancher, that grants
get
permission on the non-resource URL/metrics
endpoint.As an admin user, generate an un-scoped Rancher API token, and execute the following API request via cURL, to create the required role. You will need to set CATTLE_ACCESS_KEY, CATTLE_SECRET_KEY and RANCHER_URL to reflect the generated API token and your Rancher URL. You can also edit the role name, as desired, which is set to
kube-api metrics
in this example.export CATTLE_ACCESS_KEY=token-8jn92 export CATTLE_SECRET_KEY=l2r4nq9sx6pdhpm4bgwntvgk49qn6rvvmtsvlvkmjk9rjsfd7n65fz export RANCHER_URL=rancher.example.com curl -k -u "${CATTLE_ACCESS_KEY}:${CATTLE_SECRET_KEY}" \ -X POST \ -H 'Accept: application/json' \ -H 'Content-Type: application/json' \ -d '{"context":"cluster","clusterCreatorDefault":false," projectCreatorDefault":false,"name":"kube-api metrics","rules":[{"nonResourceURLs":["/metrics"],"type":"/v3/schemas/policyRule","verbs":["get"]}]}' \ "https://${RANCHER_URL}/v3/roletemplates"
-
After creating the cluster role, you can then grant this for a user or group. To do so, follow the steps in the Rancher documentation on assigning a cluster role to a user or group.
-
Once the role is granted to a user, they will be able to test their access to the
/metrics
endpoint.The user can access the endpoint, with the applicable cluster id, via the Rancher proxied Kubernetes API Server endpoint, by generating a cluster-scoped or un-scoped API token. The user will need to set CATTLE_ACCESS_KEY, CATTLE_SECRET_KEY, RANCHER_URL and CLUSTER_ID to reflect the generated API token, Rancher URL and cluster id.
export CATTLE_ACCESS_KEY=token-8jn92 export CATTLE_SECRET_KEY=l2r4nq9sx6pdhpm4bgwntvgk49qn6rvvmtsvlvkmjk9rjsfd7n65fz export RANCHER_URL=rancher.example.com export CLUSTER_ID=c-wwdjc curl -k https://${RANCHER_URL}/k8s/clusters/${CLUSTER_ID}/metrics \ -H "Authorization: Bearer ${CATTLE_ACCESS_KEY}:${CATTLE_SECRET_KEY}"
For Rancher-provisioned Kubernetes clusters with Authorized Cluster Endpoint enabled, the user can also query the endpoint by connecting to the Kubernetes API Server on the cluster's controlplane nodes directly, using a cluster-scoped API token. The user will need to set CATTLE_ACCESS_KEY, CATTLE_SECRET_KEY and AUTHORIZED_ENDPOINT_ADDRESS to reflect the generated API token, and the authorized endpoint address.
export CATTLE_ACCESS_KEY=token-d6cls export CATTLE_SECRET_KEY=b6gk6lmgrhsb4rjccktzkwxn5df7tm87msggq87lpmls2pkbpc5t5r export AUTHORIZED_ENDPOINT_ADDRESS=controlplane-01.example.com curl -k https://${AUTHORIZED_ENDPOINT_ADDRESS}:6443/metrics \ -H "Authorization: Bearer ${CATTLE_ACCESS_KEY}:${CATTLE_SECRET_KEY}"
Further reading
Disclaimer
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000020052
- Creation Date: 06-May-2021
- Modified Date:06-May-2021
-
- SUSE Rancher
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com