👋 Hi! I’m Bibin Wilson. In each edition, I share practical tips, guides, and the latest trends in DevOps and MLOps to make your day-to-day DevOps tasks more efficient. If someone forwarded this email to you, you can subscribe here to never miss out!

✉️ In Today’s Newsletter

In todays deep dive, I break down everything about Kubernetes Swap.

• How Kubernetes Swap Works

• Key Use Cases for Swap

• Pod QoS Classes and Swap Behavior

• How Swap is Allocated for Pods

• Swap & Memory Limit

• Hands-on: Testing Swap on Worker Nodes

• Identifying Swap-Enabled Nodes using NFD.

• Swap Observability Using Metrics

Also in this edition,

• Argo CD Image Updater

• Complete Gitlab Architecture

• kubectl native aliases with kuberc

• Current status of AI in DevOps / SRE

🧰 Remote Job Opportunities

1. Proxify AB - Senior DevOps Engineer(AWS)

2. Nvidia - DevOps and Automation Engineer

3. Tech prescient - DevOps Engineer

4. Shure - DevOps Engineer

5. Learneo(Quillbot) - DevOps Engineer

6. Winemore - DevOps Engineer

7. Kodify Media Group - Site Reliability Engineer with Node.js

What is Kubernetes Swap?

When you deploy a Pod in Kubernetes, each node uses its physical RAM to run containers.

If a node runs out of memory, Kubernetes may start killing Pods to keep the node stable, even if some of those Pods havent crossed their own memory limits.

With kubernetes swap, you can avoid this by allowing the node to use a part of its disk as extra virtual memory.

It is based on Linux swap, but Kubernetes adds its own control through the kubelet and container runtime to manage how pods use it.

Historically, Kubernetes required swap to be disabled on all nodes (kubelet wouldn't start with swap enabled).

However, Kubernetes now has support (in beta as of version 1.34) to allow using swap on nodes.

🧱 Use Case

The key use case for swap is for apps with large memory footprint but low active usage.

For example, tools like Jenkins or SonarQube run on the Java Virtual Machine (JVM), which reserves a large amount of memory when it starts. Even if the app doesn’t fully use that memory, the space stays allocated.

In such cases, swap can help by moving the unused or old parts of memory (like inactive Java objects) to disk.

Another example is applications that may occasionally experience short-term spikes. It can use swap as a safety buffer instead of being OOMKilled. For example, batch processing jobs, report generation systems, or data import/export operations.

On resource-limited nodes (like edge devices or small VMs), swap provides a bit of extra breathing room for workloads when physical memory runs out.

Even legacy Java-based apps such as Apache Tomcat, WebLogic, or JBoss can benefit from swap in the same way.

How Swap Works in Kubernetes

When a node’s RAM is full, the kernel decides to move less-used memory pages (blocks of memory) from RAM to the swap space on disk.

If swap support is enabled on the nodes, kubelet tells the container runtime (like containerd or CRI-O) how much swap each pod is allowed to use.

In simple terms, the kubelet sets the swap rules, and the Linux kernel applies them when memory is required. This helps prevent the system from crashing due to OOM (Out Of Memory) errors by providing a temporary buffer.

Swap also comes with a trade-off. The disk I/O is much slower than RAM, so performance can drop when the system starts using swap.

Another key thing to note is, the Kubernetes scheduler ignores swap when deploying workloads. This means the scheduler only deploys pods based on requests/limits, not swap.

Pod QoS Classes & Swap Eligibility

When swap is enabled in Kubernetes, whether a pod can use swap depends on its QoS class.

Guaranteed pods and BestEffort pods cannot use swap (they are prohibited).

Burstable pods are the only ones allowed swap using the calculation given below.

Swap Memory Allocation for Pods

A pod cannot use all swap memory because there is a swap limit on the pods.

Swap memory for each pod is allocated using the formula below.

containerSwapLimit = (containerMemoryRequest / nodeTotalMemory) × totalPodsSwapAvailable

For example, lets say your node has 2GB RAM, 2GB swap memory, and the pod request is 512MiB.

Then the total swap memory the pod can use is ~512MiB.

Lets look at another example involving two Burstable pods.

Let’s say a node has 4 GB of RAM and 2 GB of swap space.

Now there are two Burstable pods. Pod A requests 1 GB and Pod B requests 2 GB.

Together they use 1.5 GB of swap, leaving 0.5 GB swap unused.

Swap & Memory Limit

Even when swap is enabled on a node, a pod can never use more memory than its defined memory.limit specified in resource spec.

Here is what happens.

The memory limit in the pod spec defines the total memory (RAM + swap) the container can use. Once a container crosses that limit, the kubelet will OOMKill (Out-Of-Memory kill) the container even if there is unused swap space.

Swap does not extend or bypass the limit.

Enabling Swap in Kubernetes (Hands on)

There is no built-in way to enable swap in Kubernetes. You must manually enable swap on each worker node where you want to use it.

Now, lets look at how to enable it. First, ssh in to the node that you want to enable swap.

The cgroup on the node should be version 2. You can check it using the following command.

$ stat -fc %T /sys/fs/cgroup

cgroup2fs

Then run the following commands to create a swap file and enable it.

sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
free -h

You will get the following output as below a 2GB swap memory is created.

$ free -h

       total        used        free      shared  buff/cache   available
Mem:   3.7Gi       576Mi       1.2Gi       2.0Mi       2.0Gi       3.0Gi
Swap:  2.0Gi          0B       2.0Gi

Then open the kubelet config file.

sudo vi /var/lib/kubelet/config.yaml

Now, add the swapBehavior as LimitedSwap as given below.

failSwapOn: false
memorySwap:
  swapBehavior: LimitedSwap

Save and close the config file and restart kubelet.

sudo systemctl daemon-reload

sudo systemctl restart kubelet

Now, log out of the node and run the following command to check if swap is enabled.

kubectl get nodes -o go-template='{{range .items}}{{.metadata.name}}: {{if .status.nodeInfo.swap.capacity}}{{.status.nodeInfo.swap.capacity}}{{else}}<unknown>{{end}}{{"\n"}}{{end}}'

If it is enabled for node01, you will get the following output. Where 2147479552 = ~2.0 GB of swap.

controlplane: <unknown>
node01: 2147479552

You can also use latest kubectl to check the swap usage using the following command.

$ kubectl top nodes --show-swap

NAME                     SWAP(bytes)   SWAP(%)     
Node01                   0Mi           0%          
Node02                   <unknown>     <unknown>   
Node03                   <unknown>     <unknown>   

Testing Swap

Let’s create a memory stress pod to see what happens when the nodes memory reaches the limit.

I am testing this on a node with 2GB RAM.

Given below are the current usage and available memory of the node.

$ free -h

        total        used        free      shared  buff/cache   available
            
Mem:    1.9Gi       917Mi       128Mi       3.3Mi       1.1Gi       1.0Gi
Swap:   2.0Gi        12Mi       2.0Gi

You can see it has it has 1Gi memory left, so I am going to create a stress pod which can create stress upto 1Gi to check memory swaping.

apiVersion: v1
kind: Pod
metadata:
  name: swap-demo
spec:
  restartPolicy: Never
  nodeSelector:
    kubernetes.io/hostname: swap-test-7vutqmvu9-pokx3
  containers:
  - name: stress
    image: alpine:3.20
    command: ["sh","-c"]
    args:
      - |
        set -e
        apk add --no-cache stress-ng >/dev/null
        echo "Before:"; free -h
        stress-ng --vm 1 --vm-bytes 1000M --vm-keep --timeout 90s --metrics-brief
        echo "After:"; free -h; sleep 20
    resources:
      requests:
        memory: "512Mi"
      limits:
        memory: "1000Mi"

Change the nodeSelector, if your nodes name is different.

Apply the above manifest and wait until it starts running.

$ free -h

        total        used        free      shared  buff/cache   available
Mem:    1.9Gi       1.7Gi        73Mi       6.7Mi       266Mi       184Mi
Swap:   2.0Gi       343Mi       1.7Gi

As you can see, some data from RAM has been moved to swap (inactive pages). Almost 343 MiB used. As the system was low on RAM, so the kernel has started using swap space to offload inactive memory pages.

This prevents the pod from being OOMKilled because of insufficient memory.

Identifying Nodes With Swap

You might not need all the nodes to be swap enabled. So how do we find nodes with swap?

One way is to label the the nodes with swap as part of provisioning. For example,

kubectl label node <node> swap-enabled=true

Another way is NFD. Kubernetes provides a way to detect hardware features and system configuration using the node feature discovery (NFD) addon.

If you don't have NFD installed, run the following command to install it using helm.

helm install -n node-feature-discovery --create-namespace nfd oci://registry.k8s.io/nfd/charts/node-feature-discovery --version 0.18.1

When you enable swap, kubernetes adds feature.node.kubernetes.io/memory-swap=true label automatically to the nodes and exposed using NFD.

Now, if you want to schedule pods only on swap-enabled nodes, you can use the nodeSelector feature with this label as shown below.

nodeSelector:
  feature.node.kubernetes.io/memory-swap: "true"

Monitoring swap

If you are enabling swap, you must monitor its usage.

This way, cluster operators can see how much swap is being used. This observability helps in understanding performance impacts and making decisions.

When you enable swap, Kubernetes provides metrics that you can monitor using tools like Prometheus. Following are the key metrics that provides observability for swap.

• node_memory_SwapCached_bytes

• node_memory_SwapTotal_bytes

• node_memory_SwapFree_bytes

Also, machine_swap_bytes shows the swap capacity of the node.

Here is a sample Grafana dashboard we created for swap using these metrics.

Thats a wrap!

While this feature helps in few use cases, you need to test it and configure it in a way it doesn't affect you workloads in the cluster.

Also, for production, in moder cloud-native workloads do not use swap unless you have a very specific, tested reason.

Argo CD Image Updater

Argo CD Image Updater is an additional tool in Argo CD that helps to automate the process of updating new container images on applications managed by Argo CD.

Image Updater supports popular container registries like AWS ECR, Docker Hub, GCR, etc, and can even access their private repositories with proper authentication.

👉 Read the complete hands-on here here

Complete Gitlab Architecture

GitLab is a popular DevOps tool that combines version control, CI/CD, and project management tools.

It is a end to end platform that can handle the complete software development lifecycle. It is a perfect fit for organisations that needs a unified platform instead of multiple tools.

Companies like Agoda, Airbus, CERN, Goldman Sachs, and NVIDIA use GitLab for their CI/CD and DevSecOps implementations.

👉 Read the detailed architecture here

Kubectl Aliases with Kuberc

Before Kubernetes v1.33, we had to manually add kubectl aliases in .bashrc or .zshrc, mixing them with other shell configurations.

This was not a native solution. Starting with Kubernetes v1.33, we can now use a dedicated YAML file called kuberc to define aliases. Kubectl directly reads these aliases from the file, making configuration much cleaner

👉 Read the complete hands-on here here

📦 Where we are: AI in DevOps / SRE

The term AIOps (AI for IT operations) is already mainstream. Many teams use AI or ML models to detect anomalies, predict outages, and assist in operations.

However, tools are not perfect yet. Experiments show that current AI agents can only solve ~13.8% of SRE scenarios in a benchmark set (ITBench) without human help.

Also, SRE / DevOps conferences report repeatedly that automation and telemetry are growing, but human judgment is still important.

When it comes to developer productivity,

In a recent Google survey, 90% of software developers now use AI tools in their workflow, spending a median of 2 hours daily.

However, many organizations still struggle to integrate AI smoothly into existing DevOps toolchains, and merging security (DevSecOps) is a known weak point.

2 Ways I Can Help You

1. Kubernetes & CKA Course: Master Kubernetes with real-world examples and achieve CKA Certification with my Comprehensive Course.

2. CKA Exam Practice Questions & Explanations: 80+ practical, exam-style scenarios designed to help you pass the CKA exam with confidence.