Kubernetes StatefulSet
Learn how Kubernetes StatefulSet provides stable Pod identity, persistent storage, stable DNS, ordered deployment, scaling and rolling updates for stateful applications.
BitCodeMatrix
Kubernetes StatefulSet Explained
Stable Identity, Storage and Ordered Deployment
A practical guide for Kubernetes, DevOps and Cloud Engineers
Introduction
Kubernetes is often introduced with stateless workloads: create several interchangeable Pods, expose them through a Service, and replace any failed Pod with another equivalent Pod. That model works extremely well for many web applications and APIs.
Stateful applications introduce a different requirement. An individual instance may need a stable identity, a stable network name, and storage that belongs to that instance. A database member, distributed system node or other stateful process may not be interchangeable with every other instance.
Kubernetes StatefulSet is the workload API object designed for this class of workload. It manages a group of Pods while maintaining a sticky identity for each Pod and providing guarantees around uniqueness and ordering. Kubernetes documentation identifies stable unique network identifiers, stable persistent storage, ordered deployment and scaling, and ordered automated rolling updates as key use cases.
1. What Is a Kubernetes StatefulSet?
A StatefulSet manages a set of Pods that require consistent identities. Unlike a Deployment, whose replicas are normally interchangeable, StatefulSet Pods have identities that persist across rescheduling.
2. StatefulSet vs Deployment
Choosing StatefulSet should be driven by application requirements, not by the fact that an application happens to write files. If stable identity, stable storage association, or ordered behavior is not required, a Deployment is often a better fit.
3. When Should You Use a StatefulSet?
StatefulSet is useful when one or more of these requirements matter:
- Stable, unique network identifiers.
- Stable, persistent storage associated with each Pod.
- Ordered, graceful deployment and scaling.
- Ordered, automated rolling updates.
- An application where members have meaningful identities rather than being fully interchangeable.
Examples can include database clusters, distributed coordination systems and other clustered software where individual members have identity or storage requirements. The application itself still needs to provide its own clustering, replication, failover or data-consistency semantics; StatefulSet does not turn an arbitrary application into a distributed database.
4. StatefulSet Architecture
Kubernetes API
|
v
StatefulSet
name: web
replicas: 3
|
+---------------+---------------+
| | |
v v v
web-0 web-1 web-2
| | |
v v v
PVC web-0 PVC web-1 PVC web-2
| | |
v v v
Persistent Persistent Persistent
Volume Volume Volume
Headless Service
web-svc
|
+-----------+-----------+
| | |
web-0 web-1 web-2
DNS DNS DNS
The important relationship is that each ordinal Pod has its own identity and, when volumeClaimTemplates are used, its own PVC. A Headless Service provides the DNS domain used to address the individual members.
5. StatefulSet Pod Identity
A StatefulSet Pod identity consists of an ordinal, a stable network identity and stable storage. The identity remains associated with the Pod even if the Pod is rescheduled onto a different node.
The exact fully qualified DNS name depends on the namespace and cluster domain. Kubernetes uses cluster.local by default unless the cluster is configured differently.
6. Ordinal Indexes
For a StatefulSet with N replicas, the default ordinal sequence is 0 through N-1. Kubernetes also adds an ordinal-related Pod label in current stable releases.
replicas: 3
web-0
web-1
web-2
Modern Kubernetes also supports configuring a non-zero starting ordinal through spec.ordinals.start. The start-ordinal feature is stable and enabled by default in Kubernetes v1.31 and later.
spec:
replicas: 3
ordinals:
start: 5
# Pods:
web-5
web-6
web-7
7. Why StatefulSet Uses a Headless Service
StatefulSet requires a Headless Service for the stable network identity of its Pods. A Headless Service is created by setting clusterIP: None.
apiVersion: v1
kind: Service
metadata:
name: web
spec:
clusterIP: None
selector:
app: web
ports:
- name: web
port: 80
The Headless Service does not provide a normal cluster virtual IP. Instead, it establishes the DNS domain under which individual StatefulSet members can be addressed.
8. StatefulSet DNS and Stable Network Identity
StatefulSet: web
Service: web
Namespace: production
Pod:
web-0
Stable DNS form:
web-0.web.production.svc.cluster.local
Within the same namespace, applications can often use the shorter service-qualified name such as web-0.web. The full DNS name is useful when explaining the Kubernetes DNS structure.
If web-0 is deleted and recreated, the replacement Pod keeps the StatefulSet identity web-0, even though its Pod IP can change. This is why clients should use the stable DNS identity rather than caching a Pod IP as the long-term identity.
9. Stable Persistent Storage
A StatefulSet can use volumeClaimTemplates to create a PVC for each Pod. The PVC is associated with the Pod's identity, allowing a replacement Pod with the same ordinal to use the corresponding persistent storage.
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes:
- ReadWriteOncePod
storageClassName: my-storage-class
resources:
requests:
storage: 10Gi
The storage class must support provisioning, or an appropriate PersistentVolume must already exist. In current Kubernetes documentation, ReadWriteOncePod is recommended over ReadWriteOnce for production examples where appropriate.
10. How volumeClaimTemplates Creates Per-Pod PVCs
StatefulSet: web
volumeClaimTemplates:
name: data
Pods:
web-0
web-1
web-2
PVCs:
data-web-0
data-web-1
data-web-2
The naming pattern is derived from the volumeClaimTemplate name and the StatefulSet Pod identity. Each Pod receives its own PVC for the template.
11. StatefulSet PVC Retention Policy
Current Kubernetes versions provide spec.persistentVolumeClaimRetentionPolicy to control PVC deletion behavior when the StatefulSet is deleted or scaled down. This feature is stable since Kubernetes v1.32 and is enabled by default.
spec:
persistentVolumeClaimRetentionPolicy:
whenDeleted: Retain
whenScaled: Retain
Use Delete carefully for data-bearing workloads. PVC deletion can lead to deletion of underlying storage depending on the PersistentVolume and storage provisioner configuration.
12. Complete StatefulSet Example
apiVersion: v1
kind: Service
metadata:
name: web
spec:
clusterIP: None
selector:
app: web
ports:
- name: web
port: 80
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web
spec:
serviceName: web
replicas: 3
selector:
matchLabels:
app: web
podManagementPolicy: OrderedReady
updateStrategy:
type: RollingUpdate
template:
metadata:
labels:
app: web
spec:
terminationGracePeriodSeconds: 30
containers:
- name: web
image: nginx:1.27
ports:
- name: web
containerPort: 80
volumeMounts:
- name: data
mountPath: /usr/share/nginx/html
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes:
- ReadWriteOncePod
storageClassName: my-storage-class
resources:
requests:
storage: 1Gi
The example demonstrates the core building blocks: a Headless Service, StatefulSet identity, ordered Pod management, a rolling update strategy and a per-Pod PVC.
13. StatefulSet Selector and Pod Labels
The StatefulSet selector must match the labels on the Pod template. If the selector does not match the template labels, StatefulSet creation fails validation.
spec:
selector:
matchLabels:
app: web
template:
metadata:
labels:
app: web
14. Ordered Deployment: OrderedReady
OrderedReady is the default Pod management policy. For a StatefulSet with three replicas, Kubernetes creates Pods in order: web-0, web-1, then web-2.
Create:
web-0 -> Running and Ready
|
v
web-1 -> Running and Ready
|
v
web-2 -> Running and Ready
The controller waits for each predecessor to become Running and Ready before proceeding to the next Pod. If an earlier Pod is unhealthy, later Pods may not be created.
15. Ordered Termination and Scaling Down
When an OrderedReady StatefulSet is scaled down, Pods are terminated in reverse ordinal order.
replicas: 3
Scale to 1:
web-2 -> terminate
web-1 -> terminate
web-0 -> remains
Before terminating a Pod, its successors must be completely shut down. This ordering is useful for applications whose membership or shutdown sequence has meaning.
16. Parallel Pod Management Policy
StatefulSet can relax ordering for Pod creation and termination by setting podManagementPolicy to Parallel. This preserves the identity guarantees but allows the controller to launch or terminate Pods without waiting for predecessor readiness or complete termination.
spec:
podManagementPolicy: Parallel
Parallel is useful when the application does not require ordered startup or shutdown. For rolling updates, additional update-strategy behavior still matters.
17. StatefulSet Update Strategies
spec:
updateStrategy:
type: OnDelete
OnDelete can be useful when an application requires explicit control over when each member is restarted.
18. StatefulSet Rolling Updates
RollingUpdate is the default update strategy. StatefulSet updates are different from Deployment rollouts because StatefulSet identity and ordering are preserved.
Initial:
web-0 v1
web-1 v1
web-2 v1
Update image to v2
Rolling replacement:
web-2 -> v2
web-1 -> v2
web-0 -> v2
The controller updates Pods in reverse ordinal order for a RollingUpdate. It waits for the updated Pod to be Running and Ready before proceeding to the next Pod under the normal OrderedReady behavior.
19. StatefulSet RollingUpdate Partition
The RollingUpdate strategy supports a partition value. Partitioning lets you control which ordinal range is updated.
spec:
updateStrategy:
type: RollingUpdate
rollingUpdate:
partition: 2
With replicas set to 3 and partition 2, Pods with ordinals less than 2 remain at the previous revision while Pods with ordinal 2 and above are updated. This can be useful for staged updates and canary-like operational patterns at the Pod ordinal level.
20. maxUnavailable for StatefulSet Rolling Updates
Current Kubernetes StatefulSet RollingUpdate also supports maxUnavailable to control how many Pods can be unavailable during a rolling update. This can speed up updates when strict one-at-a-time replacement is not required.
spec:
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
When maxUnavailable is greater than 1, the controller can burst and update multiple Pods simultaneously. This should be used only when the application can tolerate the resulting concurrency and ordering behavior.
21. StatefulSet Revision History
StatefulSet maintains ControllerRevision objects to record revisions of its Pod template. These revisions support tracking and rolling-update behavior.
- kubectl get controllerrevisions
- kubectl get controllerrevision
The exact revision objects and history management are implementation details exposed through the Kubernetes API. For operational troubleshooting, inspect the StatefulSet and its Pods first, then examine ControllerRevision objects when revision-level investigation is required.
22. Deleting a StatefulSet: Important Data Considerations
Deleting a StatefulSet deletes the StatefulSet object and begins Pod termination, but the default behavior does not automatically delete the PersistentVolumeClaims associated with the StatefulSet.
kubectl delete statefulset web
- kubectl get pods
- kubectl get pvc
The storage lifecycle should be planned separately. Depending on your storage configuration, PersistentVolumes and underlying storage may have their own reclaim behavior.
23. What Happens When a StatefulSet Pod Is Deleted?
A StatefulSet maintains the desired set of identities. If web-1 is deleted, the controller creates a replacement Pod with the same ordinal identity, web-1.
Before:
web-0
web-1
web-2
Delete:
web-1
Controller replaces it:
web-0
web-1 <- replacement
web-2
The replacement Pod can receive a different Pod IP and may run on a different node, but its StatefulSet identity remains web-1. With stable storage configured, it can also reconnect to the PVC associated with that identity.
24. Rescheduling Does Not Change Stateful Identity
A Pod may be rescheduled to a different node because of node failure, eviction or other scheduling events. The StatefulSet identity remains tied to the ordinal, not to the node.
Node A
web-1 + PVC
Node A unavailable
Node B
web-1 + same identity
|
+--> same PVC association
+--> new Pod IP possible
The storage system must support the required attachment and access semantics. StatefulSet does not bypass storage-provider constraints.
25. Readiness and Ordered Stateful Applications
Readiness is especially important with OrderedReady because the controller uses Pod readiness when deciding whether it can proceed with later Pods or update operations.
readinessProbe:
tcpSocket:
port: 8080
periodSeconds: 10
failureThreshold: 3
Choose readiness checks that represent whether the application is actually able to serve its intended role. For clustered databases, this may require an application-specific readiness endpoint rather than simply checking that a process is listening.
26. Graceful Shutdown Matters
Stateful applications often need time to flush data, leave cluster membership, close connections or transfer leadership. StatefulSet documentation strongly discourages setting terminationGracePeriodSeconds to 0 for StatefulSet Pods.
spec:
template:
spec:
terminationGracePeriodSeconds: 30
The correct grace period depends on the application. The key is to provide enough time for a controlled shutdown rather than forcing immediate termination.
27. PersistentVolume, PVC and StatefulSet Relationship
StatefulSet
|
+-- volumeClaimTemplates
|
+-- PVC data-web-0
| |
| +-- PV
|
+-- PVC data-web-1
| |
| +-- PV
|
+-- PVC data-web-2
|
+-- PVThe StatefulSet creates PVCs from the templates. A StorageClass can dynamically provision PersistentVolumes, or an administrator can provide matching PersistentVolumes. The storage system ultimately determines how volumes are provisioned, attached and mounted.
28. Does StatefulSet Make a Database Highly Available?
No. StatefulSet provides identity, storage association and ordering mechanisms; it does not implement database replication, consensus, leader election or application-level failover.
29. Headless Service vs Normal Service
A stateful application can use a Headless Service for member discovery and a separate normal Service for client traffic. Kubernetes' replicated MySQL example uses this pattern: a Headless Service provides stable member DNS, while a normal Service can distribute read traffic across Ready instances.
30. Hands-On Lab: Deploy a StatefulSet
cat <<'EOF' > statefulset.yaml
apiVersion: v1
kind: Service
metadata:
name: web
spec:
clusterIP: None
selector:
app: web
ports:
- name: web
port: 80
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web
spec:
serviceName: web
replicas: 3
selector:
matchLabels:
app: web
template:
metadata:
labels:
app: web
spec:
terminationGracePeriodSeconds: 30
containers:
- name: web
image: nginx:1.27
ports:
- containerPort: 80
volumeMounts:
- name: data
mountPath: /usr/share/nginx/html
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes:
- ReadWriteOncePod
resources:
requests:
storage: 1Gi
EOF
- kubectl apply -f statefulset.yaml
Inspect the resulting resources:
- kubectl get statefulset web
- kubectl get pods -l app=web
- kubectl get pvc
- kubectl get service web
31. Lab: Observe Stable Pod Identity
kubectl get pods -l app=web -o wide
kubectl delete pod web-1
kubectl get pods -l app=web -w
Observe that a replacement Pod with the web-1 identity is created. Its IP may differ from the deleted Pod's IP.
32. Lab: Verify Per-Pod Storage
kubectl get pvc
kubectl describe pvc data-web-0
kubectl describe pvc data-web-1
kubectl describe pvc data-web-2
You should see separate PVCs corresponding to the StatefulSet members when volumeClaimTemplates is used.
33. Lab: Test StatefulSet DNS
kubectl run dns-test --rm -it --restart=Never --image=busybox:1.36 -- sh
# Inside the temporary Pod:
nslookup web-0.web
nslookup web-1.web
nslookup web-2.web
The exact DNS tooling available depends on the test image. The purpose is to verify that individual StatefulSet members can be discovered through the Headless Service DNS domain.
34. Lab: Scale a StatefulSet
kubectl scale statefulset web --replicas=5
kubectl get pods -l app=web -w
With OrderedReady, new Pods are created in ordinal order. web-3 must become Running and Ready before web-4 is created.
35. Lab: Observe Ordered Scale-Down
kubectl scale statefulset web --replicas=2
kubectl get pods -l app=web -w
Observe that higher ordinals are terminated first: web-4, then web-3, then web-2, leaving web-0 and web-1.
36. Lab: Perform a StatefulSet Rolling Update
kubectl set image statefulset/web web=nginx:1.28
kubectl get pods -l app=web -w
For an OrderedReady StatefulSet, observe the update sequence and readiness behavior. Use a controlled test environment before applying update patterns to a production database or distributed system.
37. StatefulSet Troubleshooting
38. Essential StatefulSet Debugging Commands
kubectl get statefulset web
kubectl describe statefulset web
kubectl get pods -l app=web -o wide
kubectl describe pod web-0
kubectl logs web-0
kubectl get pvc
kubectl describe pvc data-web-0
kubectl get pv
kubectl get storageclass
kubectl get service web
kubectl describe service web
kubectl get events --sort-by=.lastTimestamp
39. Common StatefulSet Mistakes
- Using StatefulSet simply because an application writes data, without first determining whether stable identity is required.
- Forgetting to create the Headless Service referenced by serviceName.
- Using an incorrect selector that does not match the Pod template labels.
- Assuming StatefulSet itself provides database replication or failover.
- Assuming a Pod IP is stable; the stable identity is the StatefulSet identity and DNS, not the Pod IP.
- Deleting PVCs casually during cleanup.
- Setting terminationGracePeriodSeconds to 0.
- Using OrderedReady when the application does not need ordering and could benefit from parallel operations.
- Updating a production stateful application without understanding its own replication and compatibility rules.
- Assuming rolling updates are always safe for a clustered database without application-level coordination.
- Ignoring storage-provider limitations such as access mode and volume attachment constraints.
- Using aggressive PVC deletion policies without a tested data-retention procedure.
40. Kubernetes StatefulSet Best Practices
- Use StatefulSet only when stable identity, storage association or ordered behavior is actually required.
- Create the required Headless Service and set StatefulSet serviceName correctly.
- Use explicit labels and selectors that are unique to the workload.
- Use a StorageClass and storage configuration appropriate for the application's durability and access requirements.
- Prefer ReadWriteOncePod for production examples when the storage platform and workload requirements support it.
- Treat PVCs and PersistentVolumes as data-bearing resources and make retention/deletion decisions deliberately.
- Define readiness probes that represent real application readiness.
- Use an appropriate termination grace period for graceful application shutdown.
- Use OrderedReady when member ordering matters; use Parallel when ordering is unnecessary.
- Use RollingUpdate only after validating that the application supports the sequence and concurrency of member replacement.
- Consider OnDelete when operators need explicit control over member updates.
- Use update partitions when a staged ordinal-based update is useful.
- Monitor the application layer in addition to Kubernetes resource health.
- Test node failure, Pod replacement, storage attachment and recovery procedures before production incidents occur.
- Document how database/application replication interacts with StatefulSet identity and rollout behavior.
41. Real-World Scenario: Stateful Database Cluster
Consider a three-member database cluster where member 0 has a special role and members 1 and 2 replicate data. The database software requires predictable member identity and persistent data.
- StatefulSet gives the members predictable identities such as db-0, db-1 and db-2.
- A Headless Service provides stable DNS identities such as db-0.db.
- volumeClaimTemplates provides separate persistent storage for each member.
- OrderedReady can ensure db-0 becomes Ready before db-1 and db-2 are created when that ordering is required by the application.
- The database software—not StatefulSet—implements replication and determines whether a member is a primary, replica or leader.
- Operational updates must account for database compatibility, replication health and quorum requirements before changing the container image.
42. StatefulSet vs Kubernetes Operator
A StatefulSet is a Kubernetes built-in workload primitive. An Operator is an application-specific controller pattern that can encode deeper operational knowledge about a system.
For complex databases and distributed systems, an Operator may complement or manage StatefulSets rather than relying on StatefulSet alone.
43. Kubernetes StatefulSet Interview Questions
What is a StatefulSet?
A Kubernetes workload resource for Pods that require stable identity, persistent storage association and/or ordered deployment and scaling.
How is StatefulSet different from Deployment?
StatefulSet maintains sticky Pod identity and supports stable storage/network identity and ordered operations; Deployment is generally used for interchangeable stateless replicas.
What is the naming pattern of StatefulSet Pods?
By default, <statefulset-name>-<ordinal>, such as web-0, web-1 and web-2.
What provides stable network identity?
A Headless Service used with the StatefulSet.
Does StatefulSet create the Headless Service automatically?
No. The user is responsible for creating the Service.
How does StatefulSet provide stable storage?
volumeClaimTemplates can create a PVC for each Pod identity.
What happens to PVCs when a StatefulSet is deleted by default?
They are not automatically deleted.
What is OrderedReady?
The default Pod management policy that provides ordered creation and termination behavior.
What is Parallel?
A Pod management policy that relaxes ordering and allows Pods to be launched or terminated in parallel.
What are StatefulSet update strategies?
RollingUpdate and OnDelete.
What is the default update strategy?
RollingUpdate.
What is partition in StatefulSet RollingUpdate?
It controls which ordinal Pods are eligible for the update, enabling staged updates.
Does StatefulSet provide database replication?
No. Replication and failover are responsibilities of the application or database system.
Can a StatefulSet Pod move to another node?
Yes. Its Pod may be rescheduled, while its StatefulSet identity remains the same.
Is the Pod IP stable?
No. Pod IPs can change. Use the stable StatefulSet DNS identity for stable addressing.
What happens if web-1 is deleted?
The StatefulSet controller creates a replacement with the web-1 identity.
44. Scenario-Based Interview Questions
Scenario 1: web-2 is stuck
A StatefulSet has replicas=3. web-0 is Ready, web-1 is Ready, but web-2 is Pending. What do you inspect?
- kubectl describe pod web-2 and Pod events.
- PVC and volume provisioning/attachment status.
- Node scheduling capacity and constraints.
- Image availability and admission errors.
- Whether a preceding Pod is actually Ready if OrderedReady dependencies are involved.
Scenario 2: Pod IP changed
web-1 was recreated after a node failure. Its IP changed. Is this a StatefulSet failure?
No. A Pod IP is not the stable identity. The StatefulSet identity web-1 and its stable DNS/storage association are the important identity mechanisms.
Scenario 3: Data disappeared after Pod recreation
A Pod was recreated and the application appears empty. What should you inspect?
- Whether the expected PVC still exists.
- Whether the replacement Pod mounted the expected PVC.
- PVC-to-PV binding and storage events.
- Storage access mode and attachment state.
- Application startup configuration and mount path.
- Whether the application itself initialized or replaced data.
45. Practical Design Exercise
Design a three-node stateful application in Kubernetes with these requirements:
- Each member must have a stable name.
- Each member must have its own 20 GiB persistent volume.
- Members must be discoverable individually through DNS.
- Members should start in order.
- Members should be terminated in reverse order.
- A container image update should roll through the members.
A good solution should contain:
- A Headless Service with clusterIP: None.
- A StatefulSet with serviceName pointing to that Service.
- replicas: 3.
- A matching selector and Pod-template labels.
- volumeClaimTemplates requesting 20 GiB.
- OrderedReady Pod management, either explicitly or by default.
- RollingUpdate as the update strategy, either explicitly or by default.
- Readiness and graceful-shutdown configuration appropriate to the application.
46. Production Readiness Checklist
47. Summary
Kubernetes StatefulSet is designed for workloads where Pods are not interchangeable. It provides stable identities, stable network identities through a Headless Service, and stable storage associations through PersistentVolumeClaims created from volumeClaimTemplates.
With the default OrderedReady policy, StatefulSet creates Pods sequentially and terminates them in reverse order. Parallel can relax these ordering guarantees. RollingUpdate is the default update strategy, while OnDelete allows explicit control over when existing Pods are replaced.
The most important operational lesson is that StatefulSet provides workload orchestration primitives, not application-level state management. Database replication, consistency, failover, backup and recovery remain responsibilities of the application, database system and surrounding platform.
48. Suggested Next Kubernetes Articles
Kubernetes Service Explained: ClusterIP, NodePort and LoadBalancer
Kubernetes DNS Explained: CoreDNS and Service Discovery
Kubernetes PersistentVolume and PersistentVolumeClaim Explained
Kubernetes StorageClass Explained: Dynamic Volume Provisioning
Kubernetes ConfigMap Explained: Managing Application Configuration
Kubernetes Secrets Explained: Managing Sensitive Configuration
Kubernetes StatefulSet vs Deployment: Which Workload Controller Should You Use?
Kubernetes Operators Explained: Automating Complex Stateful Applications
Kubernetes DaemonSet Explained: Running a Pod on Every Node
Official Kubernetes References
StatefulSets documentation: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/
StatefulSet API reference: https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/stateful-set-v1/
Run a replicated stateful application: https://kubernetes.io/docs/tasks/run-application/run-replicated-stateful-application/
Kubernetes application management documentation: https://kubernetes.io/docs/tasks/run-application/
BitCodeMatrix.com | Kubernetes Learning Series