Kubernetes Pod Quality of Service (QoS) Explained

👋 Hi! I’m Bibin Wilson. In each edition, I share practical tips, guides, and the latest trends in DevOps and MLOps to make your day-to-day DevOps tasks more efficient. If someone forwarded this email to you, you can subscribe here to never miss out!

Do you specify requests & limits for your Kubernetes pods?

Here is why it is very important.

When deploying a pod, Kubernetes assigns a QoS class to pods based on the requests and limit parameters.

• Requests: The minimum resources a pod needs to run.

• Limits: The maximum resources a pod is allowed to use.

resources:
  requests:
    memory: "256Mi"
    cpu: "250m"
  limits:
    memory: "512Mi"
    cpu: "500m"

In this example:

• The requests for CPU is set to 250 millicores (1/4 of a CPU core) and for memory at 256 MiB.

• The limits are set to 500 millicores (1/2 of a CPU core) and 512 MiB of memory.

• This means the pod needs at least a quarter of a CPU core and 256 MiB of memory to run, but it won't use more than half a CPU core and 512 MiB of memory.

What is Pod Quality of Service? (QoS)

Kubernetes pod scheduling is based on the request value to ensure the node has the resources to run the pod.

However, a node can be overcommitted if pods try to utilize all its limit ranges more than the node’s capacity.

When pods on the node try to utilize resources that are not available on the node, kubernetes uses the QoS class to determine which pod to kill first.

Pod QoS Classes

There are three Pod QoS.

1. Best Effort

2. Burstable

3. Guranteed

Lets take a look at each one with practical examples.

1. Best effort

Your pod gets the best-effort class if you do not specify any CPU/Memory requests and limits. They use whatever resource is available. Best-effort pods are low-priority pods. The best-effort pods get killed first if the node runs out of resources.

apiVersion: v1
kind: Pod
metadata:
  name: nginx-besteffort
spec:
  containers:
  - name: nginx-container
    image: nginx

In this configuration:

• There are no resource requests or limits set for the Nginx pod.

• The pod will use whatever resources are available, but there's no guarantee.

• It's the first to be affected when the cluster runs low on resources.

2.Burstable

If you set the request lower than the limit, the pod gets a burstable class. If the node runs out of resources, they're a bit more likely to be stopped than Guaranteed pods, but less likely than BestEffort pods.

Here is a practical example.

apiVersion: v1
kind: Pod
metadata:
  name: nginx-burstable
spec:
  containers:
  - name: nginx-container
    image: nginx
    resources:
      requests:
        memory: "500Mi"
        cpu: "500m"
      limits:
        memory: "2Gi"
        cpu: "2"

In this configuration:

• Nginx is guaranteed a minimum of 0.5 CPU and 500Mi of memory but can use up to 2 CPUs and 2Gi of memory if available.

• This pod will receive more resources than a BestEffort pod but is less prioritized than a Guaranteed QoS pod.

3.Guaranteed

The pod gets a guaranteed class if the request and limit values are the same. It is considered the highest priority pod and gets killed if there are no best-effort or burstable pods.

Here is a practical example.

apiVersion: v1
kind: Pod
metadata:
  name: nginx-guaranteed
spec:
  containers:
  - name: nginx-container
    image: nginx
    resources:
      requests:
        memory: "1Gi"
        cpu: "1"
      limits:
        memory: "1Gi"
        cpu: "1"

In this configuration:

• Both the memory and CPU requests and limits are the same.

• The pod is guaranteed to have 1 CPU and 1Gi of memory.

• Kubernetes will treat this pod with high priority.

Checking QoS of Existing Pods

To check the Quality of Service (QoS) class of a pod in Kubernetes, you can describe the pod.

kubectl describe pod nginx-besteffort

This will provide detailed information about the pod, including its QoS class as shown below.

Tools to Optimize Pod QoS

There are several tools available to help you manage and optimize Pod Quality of Service (QoS) in Kubernetes:

1. Prometheus: A powerful monitoring tool that integrates well with Kubernetes. It can help you collect, store, and query metrics to understand resource usage.

2. Kubernetes Resource Recommender: KRR (Kubernetes Resource Recommender) is a CLI tool for optimizing resource allocation in Kubernetes clusters. It gathers pod usage data from Prometheus and recommends requests and limits for CPU and memory.

3. kubectl top: This command-line tool shows the resource usage for nodes or pods. It’s useful for quick checks and spot checks.

4. Kubecost: A tool that provides cost monitoring and management for Kubernetes, helping you understand and optimize your resource allocation.