Kubernetes Scheduler: How Pods Are Assigned to Nodes
Learn how the Kubernetes Scheduler assigns Pods to nodes using filtering, scoring, resources, affinity, taints, tolerations and topology spread constraints.
Kubernetes Scheduler: How Pods Are Assigned to Nodes
A practical guide to scheduling, filtering, scoring, constraints, troubleshooting, and production placement
Introduction
Kubernetes does not normally require an administrator to choose a worker node for every Pod. When a Pod is created without a node assignment, the kube-scheduler watches for it and determines a suitable node. The kubelet on the selected node can then run the Pod.
Understanding scheduling is essential for diagnosing Pending Pods, designing high-availability workloads, using node affinity and taints correctly, and preparing for Kubernetes production and senior-level interviews.
1. What Is the Kubernetes Scheduler?
kube-scheduler is the default Kubernetes scheduler and runs as part of the control plane. Its job is to select an appropriate node for newly created or otherwise unscheduled Pods.
Pod created
|
v
API Server
|
v
Scheduler observes unscheduled Pod
|
+--> Filter unsuitable nodes
|
+--> Score feasible nodes
|
v
Select node
|
v
Bind Pod to node
|
v
Kubelet runs the Pod
2. Scheduling Is More Than CPU and Memory
A common misconception is that Kubernetes simply finds a node with enough free CPU and memory. Resource requests are important, but scheduling can also be affected by node selectors, node affinity, pod affinity and anti-affinity, taints and tolerations, topology spread constraints, volume-related constraints, and other scheduler plugins and policies.
3. The Scheduling Decision: Filtering and Scoring
3.1 Filtering
Filtering removes nodes that cannot run the Pod. Examples include insufficient requested resources, an unsatisfied node selector or required affinity rule, an incompatible taint, or another hard scheduling constraint.
3.2 Scoring
After filtering, the scheduler ranks the feasible nodes using active scoring rules. The highest-ranked placement is selected. The exact behavior is implemented through the scheduler's scheduling framework and plugins.
Candidate nodes
|
v
+-----------------------+
| Filtering |
| Remove infeasible |
| nodes |
+-----------+-----------+
|
v
Feasible nodes
|
v
+-----------------------+
| Scoring |
| Rank feasible nodes |
+-----------+-----------+
|
v
Selected node
4. Scheduling Framework
The Kubernetes Scheduling Framework provides extension points implemented through scheduler plugins. A scheduling attempt has a scheduling cycle followed by a binding cycle.
5. Resource Requests and Scheduling
The scheduler uses Pod resource requests when determining whether a node can accommodate the workload. For production workloads, defining realistic CPU and memory requests is therefore important.
resources:
requests:
cpu: "500m"
memory: "512Mi"
limits:
cpu: "1"
memory: "1Gi"
A node can have free physical memory while still being rejected for a Pod if the scheduler's resource accounting says the Pod's requested resources cannot fit alongside existing workload requests.
6. nodeSelector: Simple Node Placement
nodeSelector is a simple way to require that a Pod runs only on nodes having specified labels.
apiVersion: v1
kind: Pod
metadata:
name: web
spec:
nodeSelector:
workload: general
containers:
- name: web
image: nginx:1.27
kubectl label node worker-1 workload=general
kubectl get nodes --show-labels
kubectl get pod web -o wide
7. Node Affinity
Node affinity provides more expressive node-selection rules than nodeSelector. Required rules act as hard scheduling constraints; preferred rules express a preference.
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: workload
operator: In
values:
- general
Do not confuse 'preferred' with a guarantee. Preferred rules influence placement; required rules constrain feasibility.
8. Taints and Tolerations
Taints are a node-side mechanism for repelling Pods unless those Pods have matching tolerations. A toleration allows a Pod to tolerate a matching taint; it does not by itself force the Pod onto that node.
kubectl taint nodes worker-1 dedicated=database:NoSchedule
tolerations:
- key: dedicated
operator: Equal
value: database
effect: NoSchedule
Production pattern: combine a dedicated-node label/affinity rule with a taint when you want a workload to both prefer/require a particular node pool and keep unrelated Pods away.
9. Pod Affinity and Anti-Affinity
Pod affinity and anti-affinity place a Pod in relation to other Pods. Affinity can attract workloads toward matching Pods; anti-affinity can separate workloads.
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchLabels:
app: web
topologyKey: kubernetes.io/hostname
This pattern can be used to avoid putting matching replicas on the same topology domain, such as a node, when the required topology key and rules are satisfied.
10. Topology Spread Constraints
Topology spread constraints provide declarative control over how matching Pods are distributed across topology domains such as nodes or zones. They are useful for availability and balanced placement.
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: web
DoNotSchedule makes the constraint a hard scheduling requirement. ScheduleAnyway allows scheduling while preferring placements that reduce skew.
11. A Complete Example
apiVersion: apps/v1
kind: Deployment
metadata:
name: web
spec:
replicas: 3
selector:
matchLabels:
app: web
template:
metadata:
labels:
app: web
spec:
nodeSelector:
workload: general
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: web
containers:
- name: web
image: nginx:1.27
resources:
requests:
cpu: "250m"
memory: "256Mi"
12. What Happens When a Pod Cannot Be Scheduled?
If no node satisfies the Pod's scheduling requirements, the Pod remains unscheduled. A common visible symptom is STATUS=Pending.
kubectl get pods -o wide
kubectl describe pod <pod-name>
kubectl get events --sort-by=.lastTimestamp
The most useful first diagnostic is usually kubectl describe pod because the scheduler-related Events often explain why nodes were rejected.
13. Common Scheduling Failure Scenarios
14. Practical Troubleshooting Workflow
# 1. See where the Pod is
kubectl get pod <pod> -o wide
# 2. Inspect scheduling events and conditions
kubectl describe pod <pod>
# 3. Inspect nodes and their labels
kubectl get nodes --show-labels
# 4. Inspect node capacity and conditions
kubectl describe node <node>
# 5. Inspect workload configuration
kubectl get pod <pod> -o yaml
# 6. Inspect recent cluster events
kubectl get events --sort-by=.lastTimestamp
Avoid changing scheduler configuration immediately. First identify the rejected scheduling constraint. Many scheduling incidents are caused by an incorrect label, resource request, taint/toleration, affinity rule, or topology requirement.
15. Production Design Recommendations
Set realistic CPU and memory requests for production workloads.
Use labels consistently for node pools and topology domains.
Use required affinity only when placement is genuinely mandatory.
Use preferred affinity for soft placement preferences.
Use taints to protect dedicated or special-purpose nodes.
Use topology spread constraints for deliberate replica distribution.
Avoid hard anti-affinity rules unless the cluster has enough topology capacity to satisfy them.
Test scheduling rules during scaling and node-failure scenarios, not only during normal operation.
When a Pod is Pending, inspect Events before changing the manifest.
16. Interview Questions
What is kube-scheduler?
The default Kubernetes scheduler that selects nodes for newly created or unscheduled Pods.
What are the two major node-selection steps?
Filtering feasible nodes and scoring the feasible nodes.
What is the difference between nodeSelector and node affinity?
nodeSelector provides simple label matching; node affinity provides more expressive required/preferred rules.
Does a toleration force a Pod onto a tainted node?
No. It permits the Pod to tolerate a matching taint; other scheduling rules still determine placement.
Why can a Pod remain Pending even when nodes have free CPU?
Free physical CPU is not sufficient; requested resources and other scheduling constraints must also be satisfied.
What should you check first for a Pending Pod?
kubectl describe pod <pod> and its scheduling-related Events.
17. Hands-On Exercise
Create a three-node test cluster or use an existing non-production cluster. Label two nodes with workload=general, deploy three replicas, add a topology spread constraint, then add a restrictive taint to one node. Observe how the scheduling result changes.
kubectl label node <node1> workload=general
kubectl label node <node2> workload=general
kubectl get nodes --show-labels
kubectl apply -f deployment.yaml
kubectl get pods -o wide
kubectl taint nodes <node1> dedicated=special:NoSchedule
kubectl get pods -o wide
kubectl describe pod <pending-pod>
18. Summary
The Kubernetes scheduler is a control-plane component responsible for matching unscheduled Pods with suitable nodes. The scheduling decision is not simply a free-resource lookup. The scheduler evaluates feasibility and preferences through its scheduling framework and configured plugins, while workload specifications can add constraints through resources, node selection, affinity, taints/tolerations, and topology spread rules.
Related BitCodeMatrix Topics
Kubernetes Cluster Architecture
Kubernetes Control Plane Components
Kubernetes API Server
Kubernetes Kubelet Architecture
Kubernetes Pod Lifecycle
Kubernetes Node Affinity and Anti-Affinity
Kubernetes Taints and Tolerations
Kubernetes Services and Networking
Kubernetes Production Troubleshooting