Why Kubernetes alone isn't an internal developer platform

Co-authored by Eric Schabell and Lahiru De Silva

 

Key takeaways

• Kubernetes is an infrastructure orchestrator, not a developer platform. It gives platform teams fine-grained control and gives developers complexity they never asked for.

• A complete enterprise IDP spans across multiple layers, each with its own persona. Host OS, Kubernetes distribution, multi-cluster management, application platform and developer experience, from the Infrastructure Engineer up to the Application Developer.

• SUSE Rancher Prime and WSO2 Developer Platform for OpenChoreo cover complementary halves of the multiple layers. SUSE Rancher Prime governs the infrastructure fleet, cluster lifecycle, host OS and zero trust runtime security with SUSE Security while OpenChoreo layers application platform and delivery on top, so infrastructure operations and application delivery stay cleanly separated.

• Abstractions turn infrastructure into self-service. Developers should be able to describe what an application needs in their own vocabulary, and let the platform layer fulfill those requirements, hiding the infrastructure complexity underneath.

 

The developer experience gap and why an IDP matters

Kubernetes is the de facto standard for enterprise infrastructure. It is an orchestrator, not a developer platform. Its flexible API gives platform teams fine-grained control while burying application developers in unnecessary complexity. Turning infrastructure into something usable requires higher-level abstractions.

Without a dedicated developer platform, the path from code to running application runs through tickets. A developer needing a staging environment waits on operations to provision namespaces, storage and networking. They start over if requirements change. At scale, this creates bottlenecks. Developers depend on infrastructure teams for routine tasks. Platform teams get buried in requests. Kubernetes becomes an obstacle instead of a self-service tool.

Many organizations respond by building their own platform. They assemble GitOps, secrets management, observability and deployment tooling around Kubernetes, then hand-build the integration layer. That can work, but it makes the platform team the permanent owner of an integration surface that grows more expensive to maintain as components and upgrade paths pile up.

The CNCF Platform Engineering Maturity Model frames this as a progression: Provisional, Operational, Scalable and Optimizing. Teams move from ad hoc tooling toward self-service and a managed platform product. A developer portal like Backstage helps at the interface layer, but it is not the platform itself. The portal is where developers discover services; the IDP is what turns their intent into infrastructure. A complete enterprise IDP has to cover the infrastructure foundation, Kubernetes management, multi-cluster governance, application delivery and developer experience as one connected stack, not a portal bolted onto Kubernetes.

This is where SUSE Rancher Prime and WSO2 Developer Platform for OpenChoreo play complementary roles. SUSE Rancher Prime governs Kubernetes infrastructure at enterprise scale. OpenChoreo provides the application-centric abstractions developers need. Together they turn Kubernetes infrastructure into a cohesive Internal Developer Platform (IDP).

 

A five-layer model for an enterprise IDP

Transitioning from Kubernetes infrastructure to a platform-as-a-product requires a cohesive architecture that addresses the distinct layers of the platform stack. The five-layer model below illustrates how these layers work together, from the underlying infrastructure and Kubernetes management to application delivery and developer experience.

By combining the SUSE cloud native technologies at the infrastructure and Kubernetes management layers with OpenChoreo at the application and developer experience layers, organizations can establish a production-ready blueprint for an enterprise-grade Internal Developer Platform.

The diagram below maps each layer to its core function, target persona and the corresponding SUSE and WSO2 technologies.

Figure 1: The five-layer enterprise IDP model. The SUSE cloud native technology layers cover the host operating system through multi-cluster management; OpenChoreo builds the application platform and developer experience on top of that.

Together, these five layers provide a structured foundation for building an enterprise IDP, separating infrastructure operations from application delivery and developer experience while maintaining consistent governance across the stack.

Layer 1: The immutable infrastructure foundation

A robust Internal Developer Platform begins with a reliable infrastructure foundation. No amount of developer-facing abstraction can compensate for infrastructure that is insecure, unstable or difficult to maintain. Traditional general-purpose operating systems tend to add attack surface, patching overhead and complexity that distributed orchestration environments don't need.

SUSE covers this layer with three components. SLE Micro is the immutable OS running on the Kubernetes nodes, using transactional, atomic updates to eliminate configuration drift and reduce attack surface—even in regulated or air-gapped environments. SUSE Rancher Elemental extends that management down to bare-metal provisioning, letting platform teams deploy and manage SLE Micro across hybrid and edge locations through declarative workflows instead of manual setup. SUSE Virtualization, built on Harvester and KubeVirt, runs VMs alongside containers on the same Kubernetes infrastructure, so legacy workloads don't block modernization.

 

Layer 2: The hardened orchestrator

The next layer is the Kubernetes distribution, running directly on the immutable host OS. This layer stays largely hidden from application developers in a mature IDP, who consume higher-level platform abstractions instead of interacting with Kubernetes primitives directly. For platform engineers, though, Kubernetes remains the core orchestration layer responsible for running workloads securely and reliably at scale while meeting enterprise compliance requirements.

SUSE covers this layer with two distributions matched to environment size. Rancher Kubernetes Engine 2 (RKE2) is the primary distribution, a security-focused, enterprise-grade Kubernetes runtime hardened by default for regulated and production environments, backed by extended lifecycle and support options that keep teams off frequent upgrade cycles. K3s handles resource-constrained and distributed edge environments, a lightweight distribution built for operational simplicity and a small footprint, letting the same platform architecture extend from data centers down to edge locations.

 

Layer 3: Multi-cluster management and governance

As organizations scale their cloud native initiatives, the "single massive cluster" anti-pattern rapidly breaks down. Enterprises naturally evolve into multi-cluster topologies to isolate environments, distribute workloads geographically to reduce latency or satisfy strict data sovereignty requirements. Managing this sprawling fleet introduces immense operational overhead.

SUSE Rancher Prime addresses this layer directly, providing centralized, multi-cluster Kubernetes lifecycle management. It acts as the global control plane for the entire infrastructure fleet, regardless of whether clusters are running on on-premises bare metal, VMware vSphere, Amazon EKS, Google GKE or Azure AKS.

Centralized authentication and zero trust security

Unified, strictly enforced identity management is a baseline requirement for any enterprise IDP. SUSE Rancher Prime centralizes authentication and Role-Based Access Control (RBAC) across the entire cluster fleet, ensuring access policies apply universally, and that offboarding an engineer immediately revokes their access across every data center and cloud region.

Beyond identity, SUSE Rancher Prime deeply integrates with SUSE Security to provide zero trust container security, runtime container firewalling, vulnerability scanning and active threat detection across the fleet. By weaving security into the multi-cluster management layer, platform teams protect workloads regardless of which physical cluster they are scheduled on.

Fleet GitOps and continuous delivery

Maintaining consistent configuration across multiple clusters is another core challenge. Fleet provides GitOps-based continuous delivery for managing cluster configurations, policies and infrastructure resources at scale, catching configuration drift and keeping the operating model repeatable and auditable.

The operations vs. developer divide

SUSE Rancher Prime provides comprehensive operational control across the enterprise Kubernetes infrastructure stack with topology-aware observability, integration with third-party UI extensions and fine-grained infrastructure control. But it's built for infrastructure operators and platform engineers, not product developers, and stopping here still leaves developers writing raw manifests and wrestling with Helm charts.

SUSE Rancher Prime solves infrastructure operations but it was not built for the application developer, which is exactly where the next layer picks up.

 

Layer 4: The application platform — OpenChoreo operational substrate

Layer 4 marks a real shift in focus: the goal is no longer to manage clusters, it's managing application platform and delivery. Developers require on-demand environment creation, automatic manifest generation from high-level contextual inputs and self-service access to logs and metrics, all without ever executing a kubectl command or viewing a raw YAML file.

This is where OpenChoreo extends the platform into the application and developer experience layers. OpenChoreo is a CNCF Sandbox project and a Kubernetes-native Internal Developer Platform that provides the higher-level abstractions, automation and governance self-service application delivery requires. It lets platform teams define standardized application patterns and workflows without building and maintaining the underlying controllers and integration logic themselves.

Layered on top of SUSE Rancher Prime, OpenChoreo creates a clear separation between application delivery and infrastructure operations. Developers express application intent through high-level abstractions, while OpenChoreo translates that intent into the Kubernetes resources and runtime operations required to build, deploy and operate applications across the underlying clusters.

The domain-centric Cell-based architecture

OpenChoreo organizes applications around independent, domain-oriented units called cells, using a Cell-Based Architecture (CBA). Each cell represents a bounded business context containing its applications, APIs, data and runtime controls, providing a clear ownership and isolation boundary for application teams.

Within OpenChoreo, this model maps to Kubernetes through Projects and namespaces. Creating a Project automatically establishes the underlying namespace and associated governance boundaries, while OpenChoreo manages visibility and traffic flow through network policies and API gateways. This lets developers work with business-level concepts while the platform handles the underlying Kubernetes isolation and networking.

The multi-plane topology

Rather than deploying all platform capabilities into every workload cluster, OpenChoreo separates platform responsibilities into distinct planes. This architecture centralizes coordination while allowing workload clusters to scale independently across the SUSE Rancher Prime fleet.

Figure 2: Modular multi-plane architecture. Independent control, data, build and observability planes with flexible deployment across single or multi-cluster environments.

This multi-plane design resolves severe operational complexities. For instance, physically separating the Build Plane from the Data Plane ensures that a massive, CPU-intensive container build triggered by a CI pipeline does not consume the node resources required by a live, customer-facing production application.

Multiple Data Planes can be registered to a single OpenChoreo Control Plane. A platform team might use SUSE Rancher Prime to provision an on-premises RKE2 cluster for sensitive data, an AWS EKS cluster for burst capacity and an edge K3s cluster for retail operations. All three are registered as Data Planes in OpenChoreo. When a developer promotes an application through an environment pipeline, OpenChoreo routes the workload to the correct physical cluster, abstracting the infrastructure topology away from the developer.

Declarative abstractions

OpenChoreo operational power comes from its declarative API with a suite of Kubernetes Custom Resource Definitions (CRDs) that act as binding contracts between the platform engineering team, application developers and the underlying clusters. Instead of making developers manually maintain dozens of environment-specific YAML blobs (Deployments, Services, HTTPRoutes, ConfigMaps, HPAs, NetworkPolicies and ServiceMonitors), OpenChoreo condenses the developer experience into two sets of abstractions.

Platform abstractions

Platform teams work with the infrastructure, governance and reusable application patterns everyone else consumes. These are available as Data Planes, Environments, Projects, Deployment pipelines and the Component types, Traits and Workflows that encode organizational standards into golden paths.

Developer abstractions

Developers, by contrast, work with a much smaller set of application-centric resources, primarily Components and Workloads, describing runtime requirements, dependencies and configuration at a high level, rather than touching Kubernetes Deployments, Services, HTTPRoutes, ConfigMaps, HPAs or NetworkPolicies directly.

This creates a clean boundary: platform teams define capabilities, policies and golden paths. Developers express application intent within them and OpenChoreo handles the translation between the two, without exposing developers to Kubernetes complexity along the way.

 

Layer 5: The developer experience and the agentic enterprise

The Experience Plane is the primary interaction layer for both human developers and AI agents, containing the interfaces through which users discover platform capabilities, initiate application workflows and interact with the underlying platform abstractions.

The Backstage-powered experience

OpenChoreo uses Backstage as its primary developer portal, providing a familiar interface for discovering services, APIs and platform capabilities. The portal exposes the golden paths defined by platform teams, so developers can create and manage applications without directly interacting with Kubernetes resources.

Through its platform integration, the portal can also provide access to application telemetry such as logs, metrics, traces and Kubernetes events, governed by each developer's platform permissions, unifying self-service across application delivery and operations.

The agentic enterprise

As AI agents become active participants in software development and operations, the developer experience is expanding beyond human-facing interfaces. OpenChoreo supports this evolution through secure Model Context Protocol (MCP) servers, letting AI agents interact with platform capabilities through structured, governed interfaces, rather than granting them broad access to Kubernetes APIs or cluster credentials. Agents work with the same high-level platform abstractions developers use, laying the foundation for agent-assisted application delivery, operational diagnostics and other platform workflows without sacrificing governance or access controls.

The platform ships with built-in, purpose-driven operational agents:

• SRE Agent: Assists in automated root cause analysis (RCA) and runtime diagnostics. By leveraging the observability plane, an engineer can query the agent in natural language: "What caused the memory spike and resulting OOM kill for the streaming service today?" The agent parses the relevant logs, metrics and Kubernetes events to synthesize an accurate response without violating RBAC boundaries.

• FinOps Agent: Optimizes costs based on budget alerts, continuously evaluating resource utilization across the Rancher-managed Data Planes.

• Architect Agent: Helps scaffold new services based on organizational best practices and existing catalog golden paths.

CLI and automation

The OpenChoreo CLI gives developers and automation workflows a programmatic interface to create, configure, deploy and manage applications directly from the command line using the same capabilities available through the portal. This also lets platform workflows plug into existing development and automation pipelines while keeping the same abstractions and governance enforced everywhere else.

 

Deployment flexibility and enterprise sovereignty

Enterprise IDPs have to operate within the realities of data sovereignty, regulatory requirements and heterogeneous infrastructure. WSO2 Developer Platform for OpenChoreo builds on standard Kubernetes primitives, while SUSE Rancher Prime provides centralized management across clusters and cloud environments. Together, they enable organizations to deploy platform capabilities where they’re needed, without sacrificing consistency or operational control.

Organizations can deploy this integrated stack in several configurations:

• Cloud: Deploy the platform and workloads on cloud environments such as AWS, Azure and Google Cloud, using Kubernetes clusters provisioned within the organization's chosen cloud infrastructure.

• Hybrid / Multi-Cloud: Keeping sensitive workloads in on-premises RKE2 clusters while offloading the control plane management overhead to the cloud, ideal for gradual migrations.

• On-Premises / Air-Gapped: Maintaining complete data sovereignty by self-hosting the entire stack (SUSE Rancher Prime and OpenChoreo) in secure, air-gapped private data centers. This ensures that proprietary data never leaves the organizational perimeter, a critical requirement for government, defense and highly regulated financial sectors.

 

Conclusion

Kubernetes provides the infrastructure foundation, but it does not by itself deliver the abstractions and self-service experience developers need. A successful platform engineering strategy therefore needs to address the different layers of the infrastructure and developer experience stack, separating infrastructure complexity from application development while maintaining the security, governance and operational control required at enterprise scale.

The point isn't that either half of this stack is sufficient alone, it's that most organizations already have Kubernetes and most are already assembling something like Layer 4 and 5 themselves, by hand, one integration at a time. The choice isn't "build vs. buy the whole thing", it's whether to keep owning that integration surface indefinitely or adopt a layer that's already built and maintained for you.

If you're running SUSE Rancher Prime today and your developers are still writing Kubernetes manifests, that's the gap OpenChoreo is built to close. You can try it against your own cluster at openchoreo.dev, read the architecture docs or reach out to discuss what a five-layer rollout looks like for your environment.