How to quarantine Kubernetes pods?

I wanted to share one quick tip on how you can quarantine your Kubernetes pods.

You would use this if you want to investigate the containers within a pod, but you don't want them to be part of the Kubernetes Service. You don't want to debug something while the traffic might still be flowing to that container.

How to do it?

A way to quarantine or remove a pod from the ReplicaSet is by updating the labels on that pod. Once you do that the ReplicaSet will not be in control of that pod anymore.

Let's look at a quick example. I have deployed a Hello World application and I have 5 pods running in my cluster:

$ kubectl get pods --show-labels
NAME                           READY   STATUS    RESTARTS   AGE
hello-world-5fd44c56d7-55hmc   1/1     Running   0          5m27s   app=hello-world,pod-template-hash=5fd44c56d7
hello-world-5fd44c56d7-66rp6   1/1     Running   0          10m     app=hello-world,pod-template-hash=5fd44c56d7
hello-world-5fd44c56d7-d8g4j   1/1     Running   0          2d23h   app=hello-world,pod-template-hash=5fd44c56d7
hello-world-5fd44c56d7-fv7wg   1/1     Running   0          10m     app=hello-world,pod-template-hash=5fd44c56d7
hello-world-5fd44c56d7-t7644   1/1     Running   0          10m     app=hello-world,pod-template-hash=5fd44c56d7

You can see from the output that there's only one label on the pods - app=hello-world. To quarantine one of the pods we need to update that label.

I'll pick the pod hello-world-5fd44c56d7-55hmc and update the value of the app label to debug:

kubectl label pod hello-world-5fd44c56d7-55hmc app=debug --overwrite

I am using the label command with the resource and the name of the resource. Next, I am specifying the label I want to add or update (app=debug) and finally, I need to specify --overwrite because label app already exists.

$ kubectl get pod --show-labels
NAME                           READY   STATUS    RESTARTS   AGE
hello-world-5fd44c56d7-2ssww   1/1     Running   0          52s     app=hello-world,pod-template-hash=5fd44c56d7
hello-world-5fd44c56d7-55hmc   1/1     Running   0          9m13s   app=debug,pod-template-hash=5fd44c56d7
hello-world-5fd44c56d7-66rp6   1/1     Running   0          14m     app=hello-world,pod-template-hash=5fd44c56d7
hello-world-5fd44c56d7-d8g4j   1/1     Running   0          2d23h   app=hello-world,pod-template-hash=5fd44c56d7
hello-world-5fd44c56d7-fv7wg   1/1     Running   0          14m     app=hello-world,pod-template-hash=5fd44c56d7
hello-world-5fd44c56d7-t7644   1/1     Running   0          14m     app=hello-world,pod-template-hash=5fd44c56d7

After you've changed the label, you will notice that ReplicaSet creates a new pod, but the pod with name hello-world-5fd44c56d7-55hmc stays around. Now you can inspect and debug the pod without impacting anything.