8 Control Plane Services and Extensions

This chapter advances the platform from foundational clusters to a usable control plane by clarifying the difference between services and extensions and showing how that distinction guides delivery and scale. Services are cluster-wide capabilities the platform operates on behalf of users, while extensions expand the Kubernetes API so developers can provision new resources through familiar manifests. The categorization matters both for a single, early-stage platform team—where two simple pipelines (services vs. extensions) accelerate delivery and reduce overlap—and at scale, where it informs team ownership and release models. The guidance is to keep extensions within the platform product team until practices mature, since they directly shape developer experience; implementation details can shift a capability from “service” to “extension,” so the boundary is defined by how users consume it, not just what it does.

For control plane services, the chapter emphasizes starting lean but production-like: deploy essentials (such as metrics-server, kube-state-metrics, and an event exporter) via a pinned, test-driven Helm-based pipeline, and validate with both smoke and functional tests. It urges enabling high availability—even in non-prod—to preserve production confidence, and recommends proof-of-concept experimentation before committing to vendors or tooling. Observability is nonnegotiable (logs, metrics, events, tracing) and must integrate with platform identity for self-serve dashboards and alerts. Deployment choices range from issuing scoped service-account credentials to adopting cluster-side deployers (e.g., GitOps controllers), which may also require on-cluster secrets solutions. The chapter cautions against ticket-driven chart mirroring and proposes automated mirroring plus in-pipeline scanning and admission controls. Stakeholders—especially security—should drive outcomes or offer APIs, not manual gates, to avoid reintroducing friction.

For control plane extensions, the chapter demonstrates a service mesh–centric ingress pattern using Istio for traffic policy, resilience, mTLS, and Gateway API alignment, complemented by cert-manager for certificates and external-dns for DNS automation; cloud access is granted through OIDC-assumable roles. A dedicated pipeline provisions roles, installs extensions, applies default gateways and issuers, and runs end-to-end tests (for example, verifying HTTPS access to an app via DNS and gateway). It advises caution when tightly coupling mesh features to the CNI to preserve evolvability and zero-trust principles. To scale developer self-serve beyond compute, the chapter highlights operators (e.g., Crossplane or cloud-native operators) for persistent data resources, while noting their distinct operational model and prudent scope. Finally, it introduces lightweight platform management APIs (teams, namespaces, integrations) to make onboarding and authorization consistent and event-driven; until those exist, a minimal, temporary pipeline can support early adopters with just enough automation.

We now have the foundational components in six of our eight primary domains.

The easiest way to understand the difference is to focus on extensions. When deployed, an extension enables developers to provision a resource through the Kubernetes API that Kubernetes does not natively support.

The actions we want to perform for each cluster amount to an effortless flow. For each cluster, we want to use Helm to deploy a pinned version of each of our services. Then, we want to run integration tests to confirm the health of each service. Our extensions pipeline will follow the same pattern.

Our services vs extensions chart lists a few of the most commonly used categories of extensions.

A service mesh acts as a control plane for proxy servers, deploying and configuring them based on our instructions, creating a standardized way of utilizing this integration point.

For our epetech starting platform, we will deploy the Istio service mesh and configure it to support the platform-managed DNS ingress pattern as a self-serve experience for our users.

The requirements for our extensions pipeline are the same as for our services pipeline. The differences being, since extension will provision some other resource it is possible the extension will need permission to interact with the cloud provider and the resources it manages.

Istiod can use the Kubernetes API permissions to create a load balancer, but cert-manager and external-dns will need specific Route53 permissions.

A “teams” API can be the interface to onboard a team and maintain the authoritative database for customer teams. This API will maintain the correct RoleBindings on the respective clusters. It can also populate the ‘sync team’ events stream to which other services can subscribe.

• Having multiple development and testing environments that the platform's developers and users do not use is critical for delivering non-disruptive change.
• Include a Preview environment where platform customers deploy their code, but that is not in their path to production.
• This significantly increases the likelihood of discovering breaking changes before they reach the general developer-facing environments.
• Control plane services collect, analyze, return, or forward data about the state and activities of the cluster or applications running on the cluster.
• Control plane extensions enable platform users to provision and configure resources, whether inside or outside of the cluster, that are generally not part of the Kubernetes API - they extend the API.
• Observability services, cluster autoscaling, deployment support, and security are the services that should be included or assessed as part of the minimum required services for general adoption of an engineering platform.
• Storage classes, Ingress or Gateway API, application-specific resiliency, and a general authentication and authorization capability are the extensions that should be included or assessed as part of the minimum required extensions for general adoption of an engineering platform.
• Maintain an architectural distinction between services and extensions
• When considering assigning the responsibility of an on-cluster capability to a team outside of the engineering platform product teams, assume that services will be the only effective candidates.
• Moving extensions to an external team carries a significant risk of introducing friction and waste into the platform experience.
• Use of a service mesh to provide the Gateway API, dynamic routing, and traffic policy implementation is the most effective means of managing domain changes (supporting evolutionary architecture) and preserving a zero-trust networking strategy.
• The pipeline testing strategy for deploying services and extensions should include both general status level checks and functional tests that use the service or extension.
• As part of your platform adoption strategy, plan on having a period where only one or two alpha customers are onboarded to the platform to provide early feedback on the user experience, before the platform is considered at MVP or general availability.
• While the use of a single, early adopter team to provide feedback during feature development is an effective strategy, only during this development phase should the feature have any non-self-serve process.
• General release of a platform can happen early, even with the MVP feature set, but from the first general release onward, all features must be fully self-serve.
• Providing developers a self-serve means of provisioning, configuring, and maintaining the routinely or commonly used infrastructure resources that reside outside the cluster, such as databases, file shares, message queues, API gateway integrations, etc, should be part of the platform roadmap, but is not necessarily MVP.
• While providing developers with Terraform (or any IaC language) starter kits can be both accelerating and necessary at the Enterprise level in general, these resources should not be part of the principal engineering platform roadmap.
• A small set of lightweight, custom Platform management API is a necessary part of maintaining the engineering platform product experience.
• The platform management APIs are also the means of enabling effective participation in the platform roadmap by teams outside the platform product ownership.

FAQ