How to run workloads on etcd or controlplane nodes, without the worker role, in a Rancher Kubernetes Engine (RKE) or Rancher v2.x provisioned Kubernetes cluster
This document (000020116) is provided subject to the disclaimer at the end of this document.
Although it is normally not advised to run workloads on your controlplane and etcd nodes, there are occasionally scenarios when this is necessary. A few common examples are virus scanning, monitoring, and log collection workloads.
- A Rancher Kubernetes Engine (RKE) or Rancher v2.x provisioned Kubernetes cluster
Both the controlplane and etcd nodes, which are not additionaly designated the worker role, have taints. When RKE or Rancher provisions these nodes, it adds these taints automatically. Workloads that need to run on these nodes require tolerations for these taints. For Rancher managed clusters you can see these taints within the Rancher UI on the cluster node view. The following kubectl command will also list the taints for each node.
$ kubectl get nodes -o custom-columns=NAME:.metadata.name,TAINTS:.spec.taints NAME TAINTS ip-10-0-2-10 [map[effect:NoExecute key:node-role.kubernetes.io/etcd value:true]] ip-10-0-2-11 [map[effect:NoSchedule key:node-role.kubernetes.io/controlplane value:true]] ip-10-0-2-12 <none>
Per this output, each etcd node has the
node-role.kubernetes.io/etcd=true and each controlplane node has the
The Rancher UI does not have fields for adding tolerations, so you will need to specify the tolerations directly in the workload's YAML manifest. You can use the
Import YAML button to deploy your workload and make sure to add the following tolerations block in your manifest:
spec: ... template: ... spec: ... tolerations: - operator: Exists ...
If you have an existing workload, you can also select the
View/Edit YAML option for the workload and apply the above change. This toleration will allow you to run the workload on any nodes with taints, so use with caution. If you are using Helm charts, you can also specify the same YAML in your Helm chart.
For more information on how taints and tolerations work in Kubernetes, see: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000020116
- Creation Date: 09-Jul-2021
- Modified Date:09-Jul-2021
- SUSE Rancher
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com