Task

This article details how to create a cluster role to grant users access to the /metrics endpoint of the Kubernetes API Server, in Rancher-managed Kubernetes clusters.

Pre-requisites

• A Rancher v2.x instance
• A Rancher-managed Kubernetes cluster, with Metrics Server deployed. This is deployed by default in Rancher-provisioned clusters.

Resolution

In Rancher v2.4+ it should be possible to define a non-resource URL grant via role creation within the Rancher UI. However, this is affected by the issue tracked in Issue #30321 and use of the Rancher v3 API is therefore required to create the role.

1. The first step is to create a custom cluster role within Rancher, that grants get permission on the non-resource URL /metrics endpoint.

   As an admin user, generate an un-scoped Rancher API token, and execute the following API request via cURL, to create the required role. You will need to set CATTLE_ACCESS_KEY, CATTLE_SECRET_KEY and RANCHER_URL to reflect the generated API token and your Rancher URL. You can also edit the role name, as desired, which is set to kube-api metrics in this example.

   export CATTLE_ACCESS_KEY=token-8jn92
   export CATTLE_SECRET_KEY=l2r4nq9sx6pdhpm4bgwntvgk49qn6rvvmtsvlvkmjk9rjsfd7n65fz
   export RANCHER_URL=rancher.example.com
   curl -k -u "${CATTLE_ACCESS_KEY}:${CATTLE_SECRET_KEY}" \
   -X POST \
   -H 'Accept: application/json' \
   -H 'Content-Type: application/json' \
   -d '{"context":"cluster","clusterCreatorDefault":false," projectCreatorDefault":false,"name":"kube-api metrics","rules":[{"nonResourceURLs":["/metrics"],"type":"/v3/schemas/policyRule","verbs":["get"]}]}' \
   "https://${RANCHER_URL}/v3/roletemplates"

2. After creating the cluster role, you can then grant this for a user or group. To do so, follow the steps in the Rancher documentation on assigning a cluster role to a user or group.

3. Once the role is granted to a user, they will be able to test their access to the /metrics endpoint.

   The user can access the endpoint, with the applicable cluster id, via the Rancher proxied Kubernetes API Server endpoint, by generating a cluster-scoped or un-scoped API token. The user will need to set CATTLE_ACCESS_KEY, CATTLE_SECRET_KEY, RANCHER_URL and CLUSTER_ID to reflect the generated API token, Rancher URL and cluster id.

   export CATTLE_ACCESS_KEY=token-8jn92
   export CATTLE_SECRET_KEY=l2r4nq9sx6pdhpm4bgwntvgk49qn6rvvmtsvlvkmjk9rjsfd7n65fz
   export RANCHER_URL=rancher.example.com
   export CLUSTER_ID=c-wwdjc
   curl -k https://${RANCHER_URL}/k8s/clusters/${CLUSTER_ID}/metrics \
   -H "Authorization: Bearer ${CATTLE_ACCESS_KEY}:${CATTLE_SECRET_KEY}"

   For Rancher-provisioned Kubernetes clusters with Authorized Cluster Endpoint enabled, the user can also query the endpoint by connecting to the Kubernetes API Server on the cluster's controlplane nodes directly, using a cluster-scoped API token. The user will need to set CATTLE_ACCESS_KEY, CATTLE_SECRET_KEY and AUTHORIZED_ENDPOINT_ADDRESS to reflect the generated API token, and the authorized endpoint address.

   export CATTLE_ACCESS_KEY=token-d6cls
   export CATTLE_SECRET_KEY=b6gk6lmgrhsb4rjccktzkwxn5df7tm87msggq87lpmls2pkbpc5t5r
   export AUTHORIZED_ENDPOINT_ADDRESS=controlplane-01.example.com
   curl -k https://${AUTHORIZED_ENDPOINT_ADDRESS}:6443/metrics \
   -H "Authorization: Bearer ${CATTLE_ACCESS_KEY}:${CATTLE_SECRET_KEY}"

Further reading

• Rancher documentation on custom roles