Announcing RKE, a Lightweight Kubernetes Installer
Today, we are announcing a new open-source project called the Rancher
Kubernetes Engine (RKE), our new Kubernetes installer. RKE is extremely
simple, lightning fast, and works everywhere.
Why a new Kubernetes installer?
In the last two years, Rancher has become one of the most popular ways
to stand up and manage Kubernetes clusters. Users love Rancher as a
Kubernetes installer because it is very easy to use. Rancher fully
automates etcd, the Kubernetes master, and worker node operations.
Rancher 1.x, however, also implements container networking. Therefore, a
failure of the Rancher management plane could disrupt the operation of
the Kubernetes cluster. Users who want to stand up Kubernetes clusters
today have many choices of installers. Two of the most popular
installers we have encountered are
- Kops is perhaps the most widely used Kubernetes installer. It is
in fact much more than an installer. Kops prepares all required
cloud resources, installs Kubernetes, and then wires up cloud
monitoring services to ensure the continuing operation of the
Kubernetes cluster. Kops is closely integrated with the underlying
cloud infrastructure. Kops works the best on AWS. Support for other
infrastructure platforms like GCE and vSphere is a work in progress.
- Kubespray is a popular standalone Kubernetes installer written
in Ansible. It can install a Kubernetes cluster on any servers. Even
though Kubespray has some degree of integration with various cloud
APIs, it is fundamentally cloud independent and can, therefore, work
with any cloud, virtualization clusters, or bare-metal servers.
Kubespray has grown to be a sophisticated project with participation
from a large community of developers.
is another Kubernetes setup tool that comes with upstream Kubernetes.
Kubeadm, however, does not yet support capabilities like HA clusters.
Even though pieces of kubeadm code are used in projects like kops and
Kubespray, kubeadm is not ready as a production-grade Kubernetes
installer. Rancher 2.0 is designed to
work with any Kubernetes clusters. We encourage users to leverage
cloud-hosted Kubernetes services like GKE and AKS. For users who want to
set up their own clusters, we considered incorporating either kops or
Kubespray into our product lineup. Kops does not suit our needs because
it does not work with all cloud providers. Kubespray is in fact very
close to what we want. We especially like how Kubespray can install
Kubernetes anywhere. In the end, we decided not to use Kubespray and
instead build our own lightweight installer for two reasons:
- We can have a simpler system by starting from scratch and take
advantage of many advances in Kubernetes itself.
- We can have a faster installer by going with a container-based
approach, just like how we installed Kubernetes in Rancher 1.6.
How RKE Works
RKE is a standalone executable that reads from a cluster configuration
file and brings up, brings down, or upgrades a Kubernetes cluster. Here
is a sample configuration file:
--- auth: strategy: x509 network: plugin: flannel ssh_key_path: /home/user/.ssh/id_rsa nodes: - address: server1 user: ubuntu role: [controlplane, etcd] - address: server2 user: ubuntu role: [worker] services: etcd: image: quay.io/coreos/etcd:latest kube-api: image: rancher/k8s:v1.8.3-rancher2 service_cluster_ip_range: 10.233.0.0/18 extra_args: v: 4 kube-controller: image: rancher/k8s:v1.8.3-rancher2 cluster_cidr: 10.233.64.0/18 service_cluster_ip_range: 10.233.0.0/18 scheduler: image: rancher/k8s:v1.8.3-rancher2 kubelet: image: rancher/k8s:v1.8.3-rancher2 cluster_domain: cluster.local cluster_dns_server: 10.233.0.3 infra_container_image: gcr.io/google_containers/pause-amd64:3.0 kubeproxy: image: rancher/k8s:v1.8.3-rancher2 addons: |- --- apiVersion: v1 kind: Pod metadata: name: my-nginx namespace: default spec: containers: - name: my-nginx image: nginx ports: - containerPort: 80
We start the file by specifying authentication strategy, network model,
and local SSH key path. The main body of the cluster configuration file
consists of the following three parts:
- The nodes section describes all the servers that make up the
Kubernetes cluster. Each node assumes one or more of the three
roles: controlplane, etcd, and worker. You can add or remove nodes
in a Kubernetes cluster by changing the nodes section and rerunning
the RKE command.
- The services section describes all the system services that run on
the Kubernetes cluster. RKE packages all system services as
- The add-ons section describes the user-level programs that run on
the Kubernetes cluster. An RKE user, therefore, can specify the
Kubernetes cluster configuration and application configuration in
the same file.
RKE is not a long-running service that can monitor and operate the
Kubernetes cluster. RKE is designed to work in conjunction with a
full-fledged container management system like Rancher 2.0 or with a
stand-alone monitoring system like AWS CloudWatch, Datadog, or Sysdig.
You can then construct your own scripts to monitor the health of RKE
RKE as an Embedded Kubernetes Installer
People who build distributed applications have to deal with backend
databases, data access layers, clustering, and scaling. Instead of using
a traditional application server, developers are beginning to use
Kubernetes as a distributed application platform:
- They use etcd as the backend database.
- They use Kubernetes Custom Resource Definition (CRD) to as the data
access layer, and they use
kubectlto perform basic CRUD
operations on their data model.
- They package their applications as containers and use Kubernetes for
clustering and scaling.
Applications built this way are shipped to customers as Kubernetes YAML
files. Customers can run these applications easily if they already have
Kubernetes clusters running or have access to a cloud-hosted Kubernetes
service like GKE or AKS. But what happens to the customers who want to
install the applications on virtualized or bare-metal servers? An
application developer can address this need by bundling RKE into the
application as an embedded Kubernetes installer. The application install
can start by invoking RKE and create a Kubernetes cluster for the
customer. We are seeing a tremendous amount of interest to embed a
lightweight installer like RKE into distributed applications.
You can download RKE
from GitHub. I encourage you
to read the blog post by
Hussein Galal, who wrote a significant portion of RKE code, for a more
in-depth introduction to RKE. Join us
tomorrow for an Online Meetup where we’ll give a demo of RKE. Please
sign up today.