Kubernetes QoS Classes: Guaranteed, Burstable and BestEffort
Kubernetes QoS Classes: Guaranteed, Burstable and BestEffort
bitcodematrix.com | Kubernetes Series
1. Introduction
When a node runs out of memory, something has to give. Kubernetes needs a way to decide which Pods are the most important to keep running and which can be sacrificed. The Quality of Service (QoS) class is one of the signals it uses. Every Pod is automatically placed into one of three classes: Guaranteed, Burstable or BestEffort. The class is never written by you directly. It is calculated from the CPU and memory requests and limits that you set.
QoS classes are easy to overlook until a production incident, when an important Pod is killed while an unimportant one survives, or the other way around. This article explains how each class is determined, what it means in practice, how it relates to eviction and the Linux OOM killer, and how to choose the right class for each workload. It builds on the previous article on CPU and memory requests and limits.
After reading it you should be able to:
State the exact rules for Guaranteed, Burstable and BestEffort.
Check the QoS class of any Pod.
Explain what happens to each class under node memory pressure.
Understand how Pod priority and QoS work together.
Pick the right class for each type of workload and troubleshoot surprises.
2. The Three Classes
Figure 1: The three QoS classes ordered by protection
2.1 Guaranteed
A Pod is Guaranteed only when every container sets CPU and memory limits and the requests match the limits. If you set only the limits and leave out the requests, Kubernetes copies the limit into the request, which also qualifies. Because the Pod can never use more than it requested, it does not compete with other Pods for memory beyond what the scheduler has reserved for it.
apiVersion: v1
kind: Pod
metadata:
name: guaranteed-demo
spec:
containers:
- name: app
image: nginx:1.27
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
cpu: 500m
memory: 512Mi
2.2 Burstable
Burstable is the broadest class. A Pod falls here if it has some resource settings but does not satisfy the Guaranteed rule. Typical examples are a Pod whose request is lower than its limit, a Pod that sets only a memory request, or a multi-container Pod in which one container is fully specified and another is not. Burstable Pods may use more than they requested when the node has spare capacity, which is what the name refers to.
apiVersion: v1
kind: Pod
metadata:
name: burstable-demo
spec:
containers:
- name: app
image: nginx:1.27
resources:
requests:
cpu: 250m
memory: 256Mi
limits:
memory: 512Mi
2.3 BestEffort
A BestEffort Pod has no requests or limits for any container. The scheduler treats it as needing nothing, so it can be placed almost anywhere, and it can use whatever happens to be free. It is the first to suffer when resources are scarce.
apiVersion: v1
kind: Pod
metadata:
name: besteffort-demo
spec:
containers:
- name: app
image: nginx:1.27
2.4 Common edge cases
Remember that init containers count too. A single init container without resource settings can prevent an otherwise well-specified Pod from being Guaranteed. Also note that a LimitRange or admission mutation that injects defaults changes the resulting class, because the class is calculated from the final Pod spec that the API server stores.
3. Checking the QoS Class
kubectl get pod guaranteed-demo -o jsonpath='{.status.qosClass}'
# Show class for every Pod in a namespace
kubectl get pods -n shop \
-o custom-columns=NAME:.metadata.name,QOS:.status.qosClass
kubectl describe pod burstable-demo | grep "QoS Class"
The class is stored in the Pod status when the Pod is created. It is immutable for the lifetime of the Pod. Changes to the resource values of a running Pod through in-place resize are not allowed to change the QoS class, so if you need a different class you must replace the Pod, which in practice means updating its controller's template.
4. What Each Class Means in Practice
4.1 Scheduling
The scheduler looks at requests, not at the QoS class. A BestEffort Pod requests nothing and therefore always "fits". This is why BestEffort Pods can silently crowd a node: they are invisible to the scheduler's accounting but still consume real memory and CPU.
4.2 CPU behaviour
CPU is compressible, so QoS classes affect CPU only through the request weight and any limit. Under contention, CPU is shared in proportion to requests. A BestEffort container, with essentially no request weight, gets the smallest share. Guaranteed Pods with a CPU limit are throttled at that limit. With the static CPU Manager policy configured on the node, Guaranteed Pods that request whole CPUs can receive exclusive cores.
4.3 Memory behaviour and the OOM killer
Memory is where QoS matters most. The kubelet sets the Linux oomscoreadj value for each container according to the Pod's class. The kernel OOM killer prefers to kill processes with higher scores.
The values above reflect the documented kubelet behaviour, and Pods with a very high priority value (system-critical Pods) are also given a protective score. Exact numbers can differ between versions and runtimes, so treat them as the general pattern rather than a contract.
Do not confuse this with a container hitting its own memory limit. That is a cgroup-level OOM kill (exit code 137, reason OOMKilled), and it happens regardless of QoS class. QoS matters when the whole node is short of memory.
5. Node-Pressure Eviction
Before the kernel OOM killer has to act, the kubelet monitors signals such as memory.available, nodefs.available and imagefs.available. When a threshold is crossed, the kubelet tries to reclaim resources by evicting Pods. Evicted Pods are terminated and their phase set to Failed, and a controller such as a Deployment creates replacements, usually on other nodes.
5.1 How the kubelet ranks Pods
This is a frequent source of confusion. According to the Kubernetes documentation, when ranking Pods for eviction on resource pressure the kubelet considers, in order:
Whether the Pod's usage of the starved resource exceeds its requests.
The Pod's priority (PriorityClass value).
How far the Pod's usage is above its request, relative to the request.
The kubelet does not rank directly by QoS class. However, the QoS class strongly influences the outcome, because a Guaranteed Pod cannot exceed its request (its limit equals its request), so it ends up at the bottom of the ranking. A BestEffort Pod has a request of zero, so any usage is above its request, which puts it among the first candidates. Burstable Pods are in between and depend on how much they exceed their requests. The documentation also notes that Guaranteed and Burstable Pods using less than their requests are evicted last, and generally only if system daemons need the resources.
5.2 Eviction thresholds
Hard eviction thresholds act immediately, for example memory.available below a fixed amount (a default of 100Mi exists for memory on many setups). Soft thresholds apply after a grace period. The values are configurable in the kubelet configuration and may differ on managed services.
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
evictionHard:
memory.available: "200Mi"
nodefs.available: "10%"
evictionSoft:
memory.available: "500Mi"
evictionSoftGracePeriod:
memory.available: "1m30s"
5.3 Recognising an evicted Pod
kubectl get pods -n shop
# STATUS shows Evicted
kubectl describe pod <pod-name> -n shop
# Status: Failed
# Reason: Evicted
# Message: The node was low on resource: memory. ...
The message text varies by version. It normally names the starved resource and, for memory, the container that was using more than its request.
6. Pod Priority and QoS Together
QoS and Pod priority are two different mechanisms that are often mixed up.
For your most important workloads, use both: make the Pod Guaranteed so its resources are reserved and predictable, and assign a high PriorityClass so it wins scheduling and preemption decisions.
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: business-critical
value: 100000
globalDefault: false
description: "Customer-facing production services"
---
apiVersion: v1
kind: Pod
metadata:
name: payments
spec:
priorityClassName: business-critical
containers:
- name: app
image: nginx:1.27
resources:
requests: { cpu: "1", memory: 1Gi }
limits: { cpu: "1", memory: 1Gi }
7. Choosing the Right Class
There is a real trade-off. Guaranteed Pods give predictability but reduce efficiency, because you reserve their full limit at all times and (with a CPU limit) cannot burst above it. Burstable Pods improve utilisation and are sufficient for most services, provided that their memory request is realistic. BestEffort should be avoided in production namespaces, and can be prevented using a LimitRange that supplies defaults or an admission policy that rejects Pods without requests.
8. Hands-On Walkthrough
This exercise creates one Pod of each class and verifies the result. Use any test cluster.
Create a namespace called qos-demo.
Apply the three Pod manifests from section 2 into that namespace.
List the class of every Pod.
Describe one Pod to see the QoS Class line.
Clean up the namespace.
kubectl create namespace qos-demo
kubectl apply -n qos-demo -f guaranteed.yaml
kubectl apply -n qos-demo -f burstable.yaml
kubectl apply -n qos-demo -f besteffort.yaml
kubectl get pods -n qos-demo \
-o custom-columns=NAME:.metadata.name,QOS:.status.qosClass
kubectl delete namespace qos-demo
Expected output (illustrative):
NAME QOS
besteffort-demo BestEffort
burstable-demo Burstable
guaranteed-demo Guaranteed
To see how a node reports its commitments, run kubectl describe node and review the Non-terminated Pods and Allocated resources sections. Deliberately triggering node memory pressure is not recommended on a shared cluster.
9. Troubleshooting
10. Best Practices
Set resource requests for every production container so that no Pod is accidentally BestEffort.
Use Guaranteed for databases and other workloads that must not be disturbed.
Use Burstable with a realistic memory request for most services, and monitor how often usage exceeds the request.
Combine QoS with PriorityClass for workloads that must be scheduled first and kept longest.
Remember init containers and sidecars when aiming for Guaranteed; every container must qualify.
Enforce defaults with LimitRanges and require requests with admission policies.
Reserve resources for the system and kubelet so that node-level pressure is less likely.
Alert on evictions, OOM kills and node memory pressure conditions.
Check the QoS class of critical Pods after every change to their resources.
11. Conclusion
QoS classes turn your requests and limits into a simple statement of importance: Guaranteed Pods have reserved, predictable resources and are the last to be reclaimed; Burstable Pods can use spare capacity and are reclaimed in proportion to how far they exceed their requests; BestEffort Pods ask for nothing and are the first to go. Remember that the kubelet ranks evictions by usage against requests and by priority, with QoS shaping the outcome indirectly, and that the kernel OOM killer uses a QoS-based score as a last resort. Set realistic requests everywhere, give critical workloads Guaranteed resources and a high priority, and use LimitRanges and quotas to keep the rest of the cluster honest.