16 Handling stateful applications with StatefulSets

This chapter explains how Kubernetes handles stateful workloads and why StatefulSets are the right tool compared to Deployments for applications that need stable identities and persistent storage. It contrasts “pets vs. cattle,” shows how StatefulSets provide stable pod names with ordinal indexes, one-or-more PersistentVolumeClaims per replica via claim templates, and stable network identities through a required headless Service. It also covers the DNS records (including SRV) that let clients and peers discover specific replicas, and illustrates all of this with a MongoDB-backed “quiz” example where each replica needs its own volume and address, plus careful initialization of a replica set. The result is predictable identity, ordered creation when needed, and scalable patterns that respect per-replica state.

The chapter then examines StatefulSet behavior in depth: at‑most‑one semantics prevent duplicate identities during node failures; replacement pods keep the same names and reclaim their volumes; and special care is required when storage is local or when volumes can’t be detached from failed nodes. Scaling up creates new pods and claims; scaling down deletes pods but typically retains claims (with a configurable retention policy) so data can be reattached later. It compares the OrderedReady and Parallel pod management policies, showing how strict ordering can avoid race conditions but may deadlock if readiness depends on quorum, and how adjusting readiness probes or policies can break such stalemates. It also shows how to import seed data safely, how Services and readiness interact for client traffic, and how to choose between one or two Services for stateful clusters.

Finally, it covers StatefulSet updates: RollingUpdate replaces pods one by one (with optional minReadySeconds) and supports partitioning for staged or canary rollouts; OnDelete gives full operator control to update pods in any order and pace. It shows how to monitor rollouts, inspect revision history via ControllerRevisions, and roll back safely. The chapter concludes by motivating Kubernetes Operators for end‑to‑end automation beyond what StatefulSets provide—demonstrating the MongoDB Community Operator, which introduces a custom resource to declare desired state, and then handles creating StatefulSets, Services, security artifacts, replica‑set initialization, and scaling logic so teams can manage complex stateful systems declaratively and reliably.

All pods from a Deployment use the same PersistentVolumeClaim and PersistentVolume.

Providing each replica with its own volume and address.

Treating entities as pets vs. treating them as cattle

A StatefulSet, its pods, and PersistentVolumeClaims

A headless Service used in combination with a StatefulSet

StatefulSets don’t delete PersistentVolumeClaims when scaling down; then they reattach them when scaling back up.

Comparison between the OrderedReady and Parallel Pod management policy

How the pods are updated over time with different update strategies

Partitioning a rolling update

Managing an application through custom resources and operators

• Stateful workloads are harder to manage than their stateless counterparts because managing state is difficult. However, with StatefulSets, managing stateful workloads becomes much easier because the StatefulSet controller automates most of the work.
• With StatefulSets you can manage a group of pods as pets, whereas Deployments treat the pods like cattle. The pods in a StatefulSet use ordinal numbers instead of random names.
• A StatefulSet ensures that each replica gets its own stable identity and its own PersistentVolumeClaim(s). These claims are always associated with the same pods.
• In combination with a StatefulSet, a headless Service ensures that each pod receives a DNS record that always resolves to the pod’s IP address, even if the pod is moved to another node and receives a new IP address.
• StatefulSet Pods are created in the order of ascending ordinal numbers and deleted in reverse order.
• The Pod management policy configured in the StatefulSet determines whether pods are created and deleted sequentially or simultaneously.
• The PersistentVolumeClaim retention policy determines whether claims are deleted or retained when you scale down or delete a StatefulSet.
• When you update the pod template in a StatefulSet, the controller updates the underlying pods. This happens on a rolling basis, from highest to lowest ordinal number. Alternatively, you can use a semi-automatic update strategy, where you delete a pod and the controller then replaces it.
• Since StatefulSets don’t provide everything needed to fully manage a stateful workload, these types of workloads are typically managed via custom API object types and Kubernetes Operators. You create an instance of the custom object, and the Operator then creates the StatefulSet and supporting objects.

FAQ