👋 Hi! I’m Bibin Wilson. In each edition, I share practical tips, guides, and the latest trends in DevOps and MLOps to make your day-to-day DevOps tasks more efficient. If someone forwarded this email to you, you can subscribe here to never miss out!

In todays deep dive, I break down how k8s works behind the scenes.

Also in this edition,

Cloudways: Cloudways Autonomous (by DigitalOcean) offers fully managed WordPress hosting on Kubernetes, with built-in scaling and high availability. Perfect for blogs and business websites.

Understanding how Kubernetes works under the hood is essential for anyone working with it.

In most Kubernetes interviews, you will be asked about cluster architecture and how core components work together for scheduling, networking, and managing Pods. If you can explain these clearly, you will stand out from other candidates

Since I can't cover everything in a single edition, I will be sending this as a three-part series. I am sure you will learn something new even if you have been working with Kubernetes for a while.

Let's get started.

The following Kubernetes architecture diagram shows all the components of the Kubernetes cluster and how external systems connect to the Kubernetes cluster.

The first and foremost thing you should understand about Kubernetes is, that it is a distributed system. Meaning, it has multiple components spread across different servers over a network.

The control plane is responsible for container orchestration and maintaining the desired state of the cluster. It has the following components.

The Worker nodes are responsible for running containerized applications. The worker Node has the following components.

Now, let’s take a deep dive into each control plane component (Part 1)

The kube-api server is the central hub of the Kubernetes cluster that exposes the Kubernetes API. It is highly scalable and can handle large number of concurrent requests.

End users, and other cluster components, talk to the cluster via the API server. Very rarely monitoring systems and third-party services may talk to API servers to interact with the cluster.

So when you use kubectl to manage the cluster, at the backend you are actually communicating with the API server through HTTP REST APIs over TLS.

Kubernetes api-server is responsible for the following.

Kubernetes is a distributed system and it needs an efficient distributed database like etcd that supports its distributed nature. It acts as both a backend service discovery and a database. You can call it the brain of the Kubernetes cluster.

etcd is an open-source strongly consistent, distributed key-value store. So what does it mean?

etcd uses raft consensus algorithm for strong consistency and availability. It works in a leader-member fashion for high availability and to withstand node failures.

For durability, etcd uses WAL. Every write is first appended to the write-ahead log on disk before being applied to the in-memory store, ensuring durability and crash recovery

So, how does etcd work with Kubernetes?

To put it simply, when you use kubectl to get kubernetes object details, you are getting it from etcd. Also, when you deploy an object like a pod, an entry gets created in etcd.

In a nutshell, here is what you need to know about etcd.

Also, etcd it is the only Statefulset component in the control plane.

The number of nodes in an etcd cluster directly affects its fault tolerance. Here's how it breaks down:

And so on. The general formula for the number of node failures a cluster can tolerate is:

Where n is the total number of nodes.

The kube-scheduler is responsible for scheduling Kubernetes pods on worker nodes.

When you deploy a pod, you specify the pod requirements such as CPU, memory, affinity, taints or tolerations, priority, persistent volumes (PV), etc. The scheduler's primary task is to identify the create request and choose the best node for a pod that satisfies the requirements.

The following image shows a high-level overview of how the scheduler works.

In a Kubernetes cluster, there will be more than one worker node. So how does the scheduler select the node out of all worker nodes?

Here is how the scheduler works.

To choose the best node, the Kube-scheduler uses filtering and scoring operations.

In filtering, the scheduler finds the best-suited nodes where the pod can be scheduled. For example, if there are five worker nodes with resource availability to run the pod, it selects all five nodes.

If there are no nodes, then the pod is unschedulable and moved to the scheduling queue. If It is a large cluster, let's say 100 worker nodes, and the scheduler doesn't iterate over all the nodes.

There is a scheduler configuration parameter called percentageOfNodesToScore. The default value is typically 50%. So it tries to iterate over 50% of nodes in a round-robin fashion. If the worker nodes are spread across multiple zones, then the scheduler iterates over nodes in different zones.

For very large clusters the default percentageOfNodesToScore is 5%.

In the scoring phase, the scheduler ranks the nodes by assigning a score to the filtered worker nodes. The scheduler makes the scoring by calling multiple scheduling plugins.

Finally, the worker node with the highest rank will be selected for scheduling the pod. If all the nodes have the same rank, a node will be selected at random.

Once the node is selected, the scheduler creates a binding event in the API server. Meaning an event to bind a pod and node.

Here is what you need to know about a scheduler.

First lets understand what is a controller?

Controllers are programs that run infinite control loops. Meaning it runs continuously and watches the actual and desired state of objects. If there is a difference in the actual and desired state, it ensures that the kubernetes resource/object is in the desired state.

Let's say you want to create a deployment, you specify the desired state in the manifest YAML file (declarative approach). For example, 2 replicas, one volume mount, configmap, etc.

The in-built deployment controller ensures that the deployment is in the desired state all the time. If a user updates the deployment with 5 replicas, the deployment controller recognizes it and ensures the desired state is 5 replicas.

Kube controller manager is a component that manages all the Kubernetes controllers. Kubernetes resources/objects like pods, namespaces, jobs, replicaset are managed by respective controllers.

Here is what you should know about the Kube controller manager.

When Kubernetes is deployed in cloud environments, the cloud controller manager acts as a bridge between Cloud Platform APIs and the Kubernetes cluster.

This way the core kubernetes core components can work independently and allow the cloud providers to integrate with kubernetes using plugins. (For example, an interface between kubernetes cluster and AWS cloud API)

Cloud controller integration allows Kubernetes cluster to provision cloud resources like instances (for nodes), Load Balancers (for services) etc.

Cloud Controller Manager contains a set of cloud platform-specific controllers that ensure the desired state of cloud-specific components (nodes, Loadbalancers, etc).

The Following are the three main controllers that are part of the cloud controller manager.

The following is a classic examples of cloud controller manager.

Deploying Kubernetes Service of type Load balancer. Here Kubernetes provisions a Cloud-specific Loadbalancer and integrates with Kubernetes Service.

Overall Cloud Controller Manager manages the lifecycle of cloud-specific resources used by kubernetes.

Envoy Gateway is an open-source project that is designed to simplify the use of Envoy Proxy as an Gateway API Controller.

Here is an interesting thing about Envoy. Compared to other Gateway API controllers, Envoy Gateway stands out because it uses Envoy Proxy is built specifically for cloud-native environments.

Envoy uses a fully dynamic control plane called xDS (Discovery Service). This means configuration changes, such as adding or updating a route config, are applied instantly in memory. There is no need to restart or reload the proxy process.

In simple terms, traffic rules can change in real time without causing downtime. This makes Envoy Gateway a strong fit for modern Kubernetes and microservices setups where changes happen often and need to be safe and fast.

👉 Read the complete hands-on here here