Kubernetes DaemonSet
Learn how Kubernetes DaemonSet runs Pods on every eligible node, handles node selection, rolling updates, scheduling, troubleshooting and production use cases.
BitCodeMatrix
Kubernetes DaemonSet Explained
Running Pods on Every Kubernetes Node
A practical guide to node-local workloads, scheduling, updates and troubleshooting
1. Introduction
Kubernetes provides several workload controllers, and each one solves a different operational problem. Deployment manages interchangeable application replicas. StatefulSet manages Pods that need stable identity. DaemonSet addresses a different requirement: a workload that should run on each node, or on each node matching a defined selection rule.
Typical DaemonSet workloads are node-local agents. Examples include log collectors, monitoring agents, security agents and cluster networking components. Kubernetes documentation describes DaemonSet as a way to ensure that all or some nodes run a copy of a Pod.
2. What Is a Kubernetes DaemonSet?
A DaemonSet is an apps/v1 Kubernetes workload resource that manages Pods intended to run on nodes. When a node becomes eligible for a DaemonSet, the controller creates the corresponding Pod. When a node is no longer eligible, the DaemonSet controller removes the Pod according to the workload's desired state.
3. DaemonSet vs Deployment
A DaemonSet is not simply a Deployment with replicas equal to the number of nodes. The controller's desired state is tied to node eligibility, so the workload automatically follows changes in the node population.
4. DaemonSet Architecture
Kubernetes API
|
v
DaemonSet
|
+----------------+----------------+
| | |
v v v
Node-1 Node-2 Node-3
| | |
Pod ds-1 Pod ds-2 Pod ds-3
If Node-4 joins and is eligible:
DaemonSet
|
+-----------------> Node-4
|
Pod ds-4
The DaemonSet controller continuously reconciles the desired state. It evaluates which nodes should run the Pod and creates or removes Pods as node eligibility changes.
5. How a DaemonSet Decides Which Nodes Receive Pods
With no node-selection constraint, a DaemonSet is intended to run on every node. You can restrict placement using standard Kubernetes scheduling constraints, such as nodeSelector or node affinity.
spec:
template:
spec:
nodeSelector:
workload: logging
Only nodes carrying the matching label are eligible for this example.
kubectl label node worker-1 workload=logging
kubectl label node worker-2 workload=logging
kubectl get nodes --show-labels
6. Complete Basic DaemonSet YAML
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: node-agent
namespace: default
spec:
selector:
matchLabels:
app: node-agent
template:
metadata:
labels:
app: node-agent
spec:
containers:
- name: node-agent
image: busybox:1.36
command:
- /bin/sh
- -c
- |
while true; do
echo "DaemonSet running on $(hostname)"
sleep 30
done
The selector must match the Pod template labels. Unlike Deployment and StatefulSet, the DaemonSet specification does not define a replicas field.
7. Why Does a DaemonSet Not Have replicas?
A Deployment asks for a number of replicas. A DaemonSet instead expresses placement: run the Pod on each eligible node. If the cluster has five eligible nodes, the desired number of DaemonSet Pods is normally five. If a sixth eligible node joins, another Pod becomes necessary.
3 eligible nodes
-> 3 DaemonSet Pods
5 eligible nodes
-> 5 DaemonSet Pods
6 eligible nodes
-> 6 DaemonSet Pods
This is why changing the node population can change the number of DaemonSet Pods without editing the DaemonSet object.
8. DaemonSet and New Nodes
One of the most useful properties of a DaemonSet is automatic coverage of newly eligible nodes. When a new node joins the cluster and satisfies the DaemonSet's placement requirements, the DaemonSet controller creates the Pod for that node.
Before:
worker-1 -> node-agent
worker-2 -> node-agent
New node:
worker-3 joins
After reconciliation:
worker-1 -> node-agent
worker-2 -> node-agent
worker-3 -> node-agent
9. DaemonSet and Node Removal
When a node is removed from the cluster, the DaemonSet no longer needs a Pod on that node. The overall desired count therefore changes with the eligible node population.
Operationally, always distinguish between deleting a Pod and deleting a node. A deleted DaemonSet Pod on an existing eligible node is normally recreated; a Pod associated with a node that no longer exists is part of the node lifecycle.
10. DaemonSet Scheduling
DaemonSet Pods are subject to Kubernetes scheduling constraints. Kubernetes documentation explains that the DaemonSet controller creates Pods on nodes according to the DaemonSet's Pod template and node-selection requirements. The controller also adds scheduling information so that DaemonSet Pods can be associated with the intended nodes.
The practical implication is important: resource requests, taints/tolerations, affinity rules, node selectors and other scheduling constraints can determine whether a DaemonSet Pod can actually run on a node.
11. Running a DaemonSet on Control-Plane Nodes
Many Kubernetes clusters taint control-plane nodes so ordinary workloads are not scheduled there. If a node-local agent genuinely needs to run on those nodes, the Pod may need an appropriate toleration.
spec:
template:
spec:
tolerations:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
The exact taint configuration varies by cluster. Inspect the actual node taints before adding tolerations.
kubectl describe node <node-name> | grep -i taint
kubectl get nodes -o custom-columns=NAME:.metadata.name,TAINTS:.spec.taints
12. Resource Requests Matter
A DaemonSet may consume resources on every eligible node. A CPU request that looks small for one Pod can become significant when multiplied across a large cluster.
The calculation is illustrative. Actual resource consumption and scheduling depend on requests, limits, node capacity and the rest of the workloads on each node.
13. Common DaemonSet Use Cases
The exact implementation depends on the product. A DaemonSet provides the Kubernetes placement mechanism; the agent itself defines what it does on the node.
14. Accessing the Host from a DaemonSet
Node-level agents sometimes need access to host filesystems, sockets or devices. This is powerful but increases security risk. For example, a logging agent may mount a host log directory.
volumeMounts:
- name: varlog
mountPath: /var/log
volumes:
- name: varlog
hostPath:
path: /var/log
15. DaemonSet Update Strategies
DaemonSet supports two update strategies: RollingUpdate and OnDelete. RollingUpdate is the default.
spec:
updateStrategy:
type: OnDelete
16. DaemonSet Rolling Updates
With RollingUpdate, changing the Pod template causes old DaemonSet Pods to be replaced with new Pods. Kubernetes documentation states that the default maxUnavailable is 1 and the default maxSurge is 0.
spec:
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 0
With maxSurge set to 0, the update does not intentionally run an additional DaemonSet Pod on a node during the replacement process. The controller replaces the existing Pod on that node.
The update process should be tested with the actual node agent because temporarily removing an agent can have operational consequences.
17. maxUnavailable
maxUnavailable controls how many DaemonSet Pods can be unavailable during a RollingUpdate. It can be expressed as an integer or percentage.
rollingUpdate:
maxUnavailable: 2
rollingUpdate:
maxUnavailable: 20%
For infrastructure agents, a conservative value may be preferable because the DaemonSet may provide observability, security or networking functions. The correct value depends on the workload's failure tolerance.
18. maxSurge
maxSurge controls how many extra DaemonSet Pods may be created above the normal number scheduled during a RollingUpdate. The default is 0.
rollingUpdate:
maxSurge: 1
Surge behavior can be useful when the node agent must be started before the old instance is removed, but it can temporarily increase resource usage. maxSurge and maxUnavailable cannot both be used to permit unrestricted simultaneous replacement; choose values appropriate for the workload.
19. minReadySeconds
DaemonSet supports minReadySeconds. It defines the minimum number of seconds a newly created Pod must remain Ready without container crashes before it is considered available.
spec:
minReadySeconds: 30
The default is 0, meaning a Pod is considered available as soon as it becomes Ready.
20. DaemonSet Revision History
DaemonSet maintains revision information using ControllerRevision objects. revisionHistoryLimit controls how many old histories are retained; the API reference documents a default of 10.
spec:
revisionHistoryLimit: 10
Revision history is useful when investigating changes and rollback behavior. ControllerRevision is primarily an internal controller mechanism, so operational workflows should generally use supported kubectl and Kubernetes APIs rather than directly manipulating revision objects.
21. DaemonSet Rollback
DaemonSets support rollback through their revision history. kubectl provides rollout commands that can inspect and undo controller revisions.
kubectl rollout history daemonset/node-agent
kubectl rollout undo daemonset/node-agent
kubectl rollout status daemonset/node-agent
A rollback should be treated as a production change: verify the resulting image and configuration, monitor the rollout, and understand whether the reverted agent is compatible with the rest of the cluster.
22. Monitoring DaemonSet Status
DaemonSet status exposes useful counts for determining whether the desired node coverage has been achieved.
kubectl get daemonset node-agent
kubectl describe daemonset node-agent
23. Useful kubectl Commands
# List DaemonSets
kubectl get daemonsets -A
# Inspect one DaemonSet
kubectl get daemonset node-agent -o yaml
kubectl describe daemonset node-agent
# Find its Pods
kubectl get pods -l app=node-agent -o wide
# Watch rollout
kubectl rollout status daemonset/node-agent
# Show revision history
kubectl rollout history daemonset/node-agent
# Inspect events
kubectl get events --sort-by=.lastTimestamp
# Inspect a node
kubectl describe node <node-name>
24. Hands-On Lab: Deploy a DaemonSet
cat <<'EOF' > daemonset.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: node-agent
spec:
selector:
matchLabels:
app: node-agent
template:
metadata:
labels:
app: node-agent
spec:
containers:
- name: node-agent
image: busybox:1.36
command:
- /bin/sh
- -c
- |
while true; do
echo "Running on $(hostname)"
sleep 30
done
EOF
kubectl apply -f daemonset.yaml
Check the result:
kubectl get daemonset node-agent
kubectl get pods -l app=node-agent -o wide
The number of Pods should correspond to the nodes eligible for the DaemonSet under the cluster's scheduling conditions.
25. Lab: Observe Node Coverage
kubectl get nodes
kubectl get pods -l app=node-agent -o wide
Compare the nodes with the nodes hosting the DaemonSet Pods. If they differ, inspect node taints, node selectors, affinity rules, resource availability and Pod events.
26. Lab: Test Pod Replacement
kubectl get pods -l app=node-agent -o wide
kubectl delete pod <daemonset-pod-name>
kubectl get pods -l app=node-agent -o wide -w
Because the node remains eligible, the DaemonSet controller should create a replacement Pod for that node.
27. Lab: Test a Rolling Update
kubectl set image daemonset/node-agent node-agent=busybox:1.37
kubectl rollout status daemonset/node-agent
kubectl get pods -l app=node-agent -o wide
Use an image tag that is available in your environment. In production, prefer controlled and verified image versions rather than casually changing infrastructure agents.
28. Lab: Roll Back
kubectl rollout history daemonset/node-agent
kubectl rollout undo daemonset/node-agent
kubectl rollout status daemonset/node-agent
Verify the resulting Pod template and image after rollback.
29. Lab: Restrict a DaemonSet to Selected Nodes
kubectl label node worker-1 agent=true
kubectl label node worker-2 agent=true
Then add this selector to the Pod template:
spec:
template:
spec:
nodeSelector:
agent: "true"
Only nodes matching the selector are eligible for the DaemonSet.
30. Troubleshooting: Pod Not Running on a Node
31. Troubleshooting Workflow
- Check the DaemonSet itself: kubectl get daemonset and kubectl describe daemonset.
- Compare desired/current/ready/updated counts.
- List DaemonSet Pods with -o wide to see node placement.
- Inspect the target node's labels and taints.
- Inspect the Pod's events for scheduling or admission errors.
- Check container logs and exit status.
- Check CPU and memory requests against node capacity.
- Check security policies if the agent uses hostPath, privileged mode, host networking or special capabilities.
- During an update, inspect rollout status and the availability of updated Pods.
32. Common DaemonSet Mistakes
- Adding replicas to a DaemonSet and expecting it to behave like a Deployment.
- Assuming every Pod runs on every node without considering taints, selectors, affinity and resource constraints.
- Using broad tolerations without understanding the security implications.
- Giving a node agent excessive host filesystem or privileged access.
- Setting high resource requests without considering multiplication across every node.
- Updating a critical logging, networking or security DaemonSet without testing its rollout behavior.
- Assuming a DaemonSet automatically means one Pod on control-plane nodes.
- Using hostPath without planning permissions and security boundaries.
- Treating DaemonSet as an application high-availability mechanism rather than a node-placement controller.
- Deleting Pods during a rollout without understanding how the controller will recreate them.
33. Production Best Practices
- Use DaemonSet for genuine node-local workloads.
- Keep selectors precise and stable.
- Define realistic CPU and memory requests because they apply across many nodes.
- Use readiness probes for agents where readiness can be meaningfully measured.
- Use minReadySeconds when the agent needs sustained readiness before being considered available.
- Use conservative maxUnavailable values for critical infrastructure agents.
- Use maxSurge only when temporary extra resource usage is acceptable.
- Pin and verify container image versions for production infrastructure components.
- Use least-privilege security settings, especially with hostPath and privileged access.
- Use tolerations only for node classes the workload genuinely needs to access.
- Monitor DaemonSet status as well as the health of the agent itself.
- Test rollback procedures before relying on them during an incident.
- Document which nodes the DaemonSet is expected to cover and why.
34. Real-World Scenario: Cluster Log Collection
Suppose a Kubernetes cluster has 20 worker nodes. Every node writes container logs locally, and the organization wants to collect those logs and send them to a centralized logging platform.
- A DaemonSet can place one log collector Pod on each eligible worker node.
- The collector can mount the node's required log directory using a carefully scoped volume.
- When a new worker node joins, the DaemonSet can place a collector there.
- When a worker node leaves, its corresponding DaemonSet Pod is no longer required.
- A RollingUpdate can progressively replace collectors when the agent image changes.
- maxUnavailable can limit how many collectors are unavailable during an update.
35. Real-World Scenario: Node Monitoring Agent
A monitoring platform needs CPU, memory, filesystem and other node-level metrics. A DaemonSet is a natural Kubernetes placement mechanism because the monitoring agent needs to run locally on each node.
The design should still distinguish between node-level metrics collected by the agent and Kubernetes/application metrics collected by other components.
36. DaemonSet and Services
A DaemonSet does not require a Service simply because it is a DaemonSet. Whether a Service is appropriate depends on how the Pods communicate.
The correct networking model depends on the workload. Do not create a Service merely because a controller manages Pods.
37. DaemonSet and Host Networking
Some infrastructure workloads may require hostNetwork, host ports or access to node-level networking. These settings should be used only when required because they reduce isolation between the Pod and the node.
spec:
template:
spec:
hostNetwork: true
38. DaemonSet vs StatefulSet
39. DaemonSet vs Static Pod
Static Pods are managed directly by the kubelet on a specific node from local manifest files or a configured static-Pod source. A DaemonSet is a Kubernetes API workload object managed through the cluster control plane and designed to place Pods across eligible nodes.
40. Interview Questions
What is a DaemonSet?
A workload controller that ensures a copy of a Pod runs on each eligible node.
Does a DaemonSet use replicas?
No. Desired Pod count is derived from eligible nodes.
What happens when a new eligible node joins?
The DaemonSet controller creates a Pod for that node.
What is the default DaemonSet update strategy?
RollingUpdate.
What is OnDelete?
A strategy where existing Pods are not automatically replaced after a template change; deleting an old Pod causes a replacement using the new template.
What is maxUnavailable?
The maximum number of DaemonSet Pods that can be unavailable during a RollingUpdate.
What is maxSurge?
The maximum number of extra DaemonSet Pods that can be created above the normal scheduled count during a RollingUpdate.
What is minReadySeconds?
The minimum time a newly created Pod must remain Ready without container crashes before being considered available.
Can a DaemonSet run on control-plane nodes?
It can if the nodes are eligible for the Pod; taints commonly require an appropriate toleration.
What is a common DaemonSet use case?
Node-local agents such as log collection, monitoring or security agents.
Can a DaemonSet have a Service?
Yes, if the application's networking design requires one, but a Service is not inherently required by DaemonSet.
Why can a DaemonSet consume significant cluster resources?
Its Pod is intended to run across many eligible nodes, so per-Pod requests multiply with node count.
Does deleting a DaemonSet Pod permanently remove it?
Normally no, if the node remains eligible; the controller reconciles the desired state and recreates it.
Does DaemonSet guarantee one Pod on every physical machine?
No. It operates on Kubernetes nodes that are eligible under the Pod's scheduling constraints.
41. Scenario-Based Interview Questions
Scenario 1: One node has no DaemonSet Pod
A cluster has 10 nodes, but only 9 have the DaemonSet Pod. What would you check?
DaemonSet status and desired/current counts.
The node's taints and labels.
nodeSelector and node affinity in the Pod template.
Pod events and scheduling errors.
Node resource availability.
Admission/security policy failures.
Scenario 2: DaemonSet rollout is stuck
A new logging-agent image was deployed and the rollout never completes.
Run kubectl rollout status daemonset/<name>.
Inspect the DaemonSet and its updated Pod counts.
Inspect failing Pods with kubectl describe pod and kubectl logs.
Check image-pull errors and resource constraints.
Check readiness probes and minReadySeconds.
Verify maxUnavailable/maxSurge are appropriate.
Scenario 3: Control-plane nodes are not covered
The DaemonSet runs on workers but not control-plane nodes.
Inspect control-plane node taints. If the agent genuinely needs to run there, add a narrowly scoped toleration that matches the actual taint. Do not remove taints merely to make the DaemonSet schedule.
Scenario 4: Resource pressure after deployment
A monitoring DaemonSet was deployed and every node now has memory pressure.
Calculate the aggregate request/usage impact per node, inspect the agent's memory behavior, review requests and limits, and determine whether the DaemonSet should run on all nodes or a smaller eligible node class.
42. Production Readiness Checklist
43. Summary
Kubernetes DaemonSet is the right workload controller when the desired state is tied to Kubernetes nodes rather than to an arbitrary replica count. It ensures that a Pod runs on each eligible node and automatically adapts as eligible nodes are added or removed.
DaemonSets are especially valuable for node-local infrastructure such as logging, monitoring, security and networking components. Their behavior is strongly influenced by Kubernetes scheduling rules, so node selectors, affinity, taints, tolerations and resource requests must be considered together.
For updates, RollingUpdate is the default strategy, while OnDelete provides manual control. maxUnavailable, maxSurge and minReadySeconds allow operators to shape rollout behavior. Because DaemonSets often run critical infrastructure agents, conservative resource, security and rollout decisions are essential.
44. Suggested Next Kubernetes Articles
- Kubernetes Service Explained: ClusterIP, NodePort and LoadBalancer
- Kubernetes DNS Explained: CoreDNS and Service Discovery
- Kubernetes PersistentVolume and PersistentVolumeClaim Explained
- Kubernetes StorageClass Explained: Dynamic Volume Provisioning
- Kubernetes ConfigMap Explained: Managing Application Configuration
- Kubernetes Secrets Explained: Managing Sensitive Configuration
- Kubernetes Job and CronJob Explained: Batch Workloads in Kubernetes
- Kubernetes HPA Explained: Horizontal Pod Autoscaling
- Kubernetes Scheduling Explained: Taints, Tolerations, Affinity and Node Selectors
Official Kubernetes References
DaemonSet concept documentation: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/
DaemonSet API reference: https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/daemon-set-v1/
Perform a rolling update on a DaemonSet: https://kubernetes.io/docs/tasks/manage-daemon/update-daemon-set/
Kubernetes API reference: https://kubernetes.io/docs/reference/using-api/
BitCodeMatrix.com | Kubernetes Learning Series