How to create a custom cluster role in Rancher v2.x to grant permission on the metrics endpoint of the kube-apiserver in a Rancher managed cluster

This document (000020052) is provided subject to the disclaimer at the end of this document.

Situation

Task

This article details how to create a cluster role to grant users access to the /metrics endpoint of the Kubernetes API Server, in Rancher-managed Kubernetes clusters.

Pre-requisites

  • A Rancher v2.x instance
  • A Rancher-managed Kubernetes cluster, with Metrics Server deployed. This is deployed by default in Rancher-provisioned clusters.

Resolution

In Rancher v2.4+ it should be possible to define a non-resource URL grant via role creation within the Rancher UI. However, this is affected by the issue tracked in Issue #30321 and use of the Rancher v3 API is therefore required to create the role.

  1. The first step is to create a custom cluster role within Rancher, that grants get permission on the non-resource URL /metrics endpoint.

    As an admin user, generate an un-scoped Rancher API token, and execute the following API request via cURL, to create the required role. You will need to set CATTLE_ACCESS_KEY, CATTLE_SECRET_KEY and RANCHER_URL to reflect the generated API token and your Rancher URL. You can also edit the role name, as desired, which is set to kube-api metrics in this example.

    export CATTLE_ACCESS_KEY=token-8jn92
    export CATTLE_SECRET_KEY=l2r4nq9sx6pdhpm4bgwntvgk49qn6rvvmtsvlvkmjk9rjsfd7n65fz
    export RANCHER_URL=rancher.example.com
    curl -k -u "${CATTLE_ACCESS_KEY}:${CATTLE_SECRET_KEY}" \
    -X POST \
    -H 'Accept: application/json' \
    -H 'Content-Type: application/json' \
    -d '{"context":"cluster","clusterCreatorDefault":false," projectCreatorDefault":false,"name":"kube-api metrics","rules":[{"nonResourceURLs":["/metrics"],"type":"/v3/schemas/policyRule","verbs":["get"]}]}' \
    "https://${RANCHER_URL}/v3/roletemplates"
  2. After creating the cluster role, you can then grant this for a user or group. To do so, follow the steps in the Rancher documentation on assigning a cluster role to a user or group.

  3. Once the role is granted to a user, they will be able to test their access to the /metrics endpoint.

    The user can access the endpoint, with the applicable cluster id, via the Rancher proxied Kubernetes API Server endpoint, by generating a cluster-scoped or un-scoped API token. The user will need to set CATTLE_ACCESS_KEY, CATTLE_SECRET_KEY, RANCHER_URL and CLUSTER_ID to reflect the generated API token, Rancher URL and cluster id.

    export CATTLE_ACCESS_KEY=token-8jn92
    export CATTLE_SECRET_KEY=l2r4nq9sx6pdhpm4bgwntvgk49qn6rvvmtsvlvkmjk9rjsfd7n65fz
    export RANCHER_URL=rancher.example.com
    export CLUSTER_ID=c-wwdjc
    curl -k https://${RANCHER_URL}/k8s/clusters/${CLUSTER_ID}/metrics \
    -H "Authorization: Bearer ${CATTLE_ACCESS_KEY}:${CATTLE_SECRET_KEY}"

    For Rancher-provisioned Kubernetes clusters with Authorized Cluster Endpoint enabled, the user can also query the endpoint by connecting to the Kubernetes API Server on the cluster's controlplane nodes directly, using a cluster-scoped API token. The user will need to set CATTLE_ACCESS_KEY, CATTLE_SECRET_KEY and AUTHORIZED_ENDPOINT_ADDRESS to reflect the generated API token, and the authorized endpoint address.

    export CATTLE_ACCESS_KEY=token-d6cls
    export CATTLE_SECRET_KEY=b6gk6lmgrhsb4rjccktzkwxn5df7tm87msggq87lpmls2pkbpc5t5r
    export AUTHORIZED_ENDPOINT_ADDRESS=controlplane-01.example.com
    curl -k https://${AUTHORIZED_ENDPOINT_ADDRESS}:6443/metrics \
    -H "Authorization: Bearer ${CATTLE_ACCESS_KEY}:${CATTLE_SECRET_KEY}"

Further reading

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:000020052
  • Creation Date: 06-May-2021
  • Modified Date:06-May-2021
    • SUSE Rancher

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback@suse.com

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

Join Our Community

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.


SUSE Customer Support Quick Reference Guide SUSE Technical Support Handbook Update Advisories
Support FAQ

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.

Go to Customer Center