The Holy Grail of PaaS on Kubernetes

Kubernetes will solve all of our problems right? It’s the container platform that will allow our development teams to deploy their microservice applications in all their cloud native glory. We’ll all be able to deploy our web apps without needing to worry about the servers that are running them or interact with the system administrators that look after those servers. Self-service for Developers!

Yeah, no. Kubernetes never promised us that. The platform was designed to orchestrate containers, and that’s what it does.

It’s essentially the next generation of IaaS, offering many of the features of platforms like Open Stack, but also a number of workload management primitives that belong a little further up the conceptual stack (Deployments, StatefulSets, Jobs, Services, Ingress, etc.). Once you get your head around these features, you realize how very powerful Kubernetes is.

But is it the answer for application developers?

Would we expect someone creating line-of-business features for a web application to know or care how their application instances are spread between availability zones? Will they want to spend the mental effort to understand how a StatefulSet is different from a DaemonSet, and which would be appropriate for their app? Just how much YAML configuration do we expect them to write outside of their core application code?

How much do we trust them to get their deployment right and not introduce a problem for other tennants on our Kubernetes cluster? Are we confident we have set up RBAC permissions so that those developers have appropriate levels of access to the API, and no more?

Is it really the right tool for a developer?

The Last Mile for App Delivery

I think the answer is no.

Some developers have enough curiosity and system administration experience to safely navigate the complexity of Kubernetes. These people are generally called DevOps Engineers, and though their numbers are growing they’re still the exception not the rule.

The majority of application developers who are primarily concerned with delivering their app will:

• enlist the help of a Kubernetes expert,
• spend a significant chunk of time learning Kubernetes (i.e. become a DevOps Engineer),
• give up, or worst of all,
• make some dangerous mistakes when deploying apps with YAML copied from StackOverflow or the Kubernetes docs

I swear this last point is not casting shade on Kubernetes. Any container platform as powerful as Kubernetes is a “foot gun” in the wrong hands.

We need something to insulate the developer from the complexity of the container platform. If we don’t want that buffer to be actual people, we need to provide an application platform that fits the needs of the developer.

This platform will need to:

• have safe multi-tennancy with granular permissions
• allow developers to work together on their applications
• provide data services rather than making the developer create their own
• let teams release rapidly and repeatably
• consume code rather than packages or container images
• give the platform operators control over policy and security

Is CI/CD the only answer?

I can’t be alone in this thinking, because a number of developer-facing tools are emerging to satisfy these requirements. Most are based on a CI/CD model: a commit or tagging event in Git triggers a build pipeline that creates new images and Kubernetes YAML (perhaps a Helm chart) which is then deployed to Kubernetes.

Like Kubernetes itself, this approach can be extremely powerful and infinitely customizable. Like Kubernetes itself, setting up the pipeline requires some expertise. The new tools are making this easier and easier, so the gap is much smaller, but there are still a couple of drawbacks to be aware of. Each application needs its own build pipeline, and the workflow is often tied to a particular source control tool, usually Git.

These are not deal breakers for everyone, but there is another approach which avoids these drawbacks, satisfies the requirements we’ve set for our developer-friendly application platform, and was designed from the start with the needs of developers in mind. As a bonus, this platform is extremely well established in our industry.

“We’ve already got one!”

The bridge between developer and CaaS is not so different from the one between developer and IaaS. Platform-as-a-Service was invented to bridge this gap, and Cloud Foundry was invented to let people operate their own platforms.

The leading edge of the IT industry sort of forgot about PaaS during all the excitement about Kubernetes, and Docker before that.

Interestingly, Docker itself has its roots in PaaS. dotCloud (as it was then called) showed off a cool technology at PyCon back in 2013 that they had created as a back end for their application platform service. We wouldn’t have the widespread interest in container platforms today if they had not invented tools to provide a multi-tenant service to developers pushing code to the cloud.

Why PaaS now, and why Cloud Foundry?

Cloud Foundry sometimes gets a bad rap amongst Kubernetes users, possibly because it wasn’t created recently and is thus seen as not very cool. It has had its own container scheduler, Diego, for years and there was a great deal of reluctance to move away from that code base. A tremendous amount of work went into creating that container orchestration, and the developers who worked on it and around it were reticent initially to investigate using somebody else’s scheduler.

It took a couple of years, but Kubernetes negated that objection. Not only was it immensely popular, but it was rapidly becoming more stable and powerful with each release. Kubernetes is now impossible to ignore.

So, a year and a half ago at the Cloud Foundry Summit in Basel, a small working group was formed by SUSE, SAP, and IBM to figure out how to use Kubernetes to power Cloud Foundry Application Runtime (CFAR). This has led to the Eirini and Quarks projects, which have generated tremendous excitement. Cloud Foundry is pivoting (pun intended, Pivotal has joined the efforts) to seize the Holy Grail of Container Schedulers.

Now it’s our turn to say “We’ve already got one!”

You see, when you replace Diego with Eirini and the BOSH Director with Quarks (a Kubernetes operator for CFAR), what you end up with is exactly what Kubernetes needs right now: a simple, safe, robust, and flexible developer interface to Kubernetes which runs natively on Kubernetes.

Developers get the Cloud Foundry experience:

here is my source code

run it on the cloud for me

I do not care how

Operators (and DevOps) get the Kubernetes experience:

here are containers

run them exactly like this

because I care how

This seems like a good separation of responsibilities, and a good way to approach software and IT transformation.