Recently I needed to work with kubernetes probes (to implement them actually) and came across some questions about them, I think it is a good idea to write a quick explanation without being too deep in the technical aspects of it but more on the why they exist and what are they used for, let's take a look.
In Kubernetes you deploy your applications in forms of images which are a single file that contains your application code and all needed dependencies for it to run.
Idealy your application will start right away and put in place to receive requests (from other containers in your cluster or from the outside world) but this some times is not so quick, your application might need some extra time to start and during that time it can't respond properly to requests, let's see an example.
Imagine your application is exposing and enpoint to the world and other applications will connect to it and send a certain payload in a POST request to that endpoint. Your application before receiving or even be able to respond to a request, it needs to do some preparations, it needs to connect to certain service to download a configuration file, then it needs to connect to another system to retrieve some secrets to connect to other services (SQL server for example) and open some connections to a database. While doing all this your application can't do much, any request you send to it will simply fail as your application is not running yet, it is initializing, so if Kubernetes tries to route traffic to it, your consummers will simply see an error. Now what?
How does traffic reach your application?
When you create a pod (wrong, you should run deployments and not pods) it has it's own IP inside the cluster, you don't send traffic directly to your pods as they can be destroyed at any time and the new ones can have a different IP, so it is not reliable to expect a pod to get traffic, instead we have services which are another Kubernetes resource which has a name (fqdn in the form ) which your other pods or ingresses should use to reach your pods, this approaches ensures you always reach your application using a reliable approach (an fqdn). Each service contains a kind of "backend pool" where your pods are assigned to receive traffic, they are assigned to this pool as soon as they are ready, and this is the tricky part, how does Kubernetes knows your applications is ready?
Startup, liveness and readiness probes
Kubernetes defines three types of probes to check the status of your pods, let's view what each one of them does and when they are executed.
Startup probes
This is the first probe to be executed (if defined) and it only runs once when the pod is started, the intention of this probe is to wait until the pod is ready to start working, but watch out, this doesn't means the application is ready to receive traffic, it means the code inside the image is ready to start working, the main intention of this probe is to wait before start running other probes (liveness and/or readiness) ensuring kubelet doesn't kill the pod before it even start operating (this is specially true in slow starting applications).
liveness probes
This is a probe that (again, if defined) will run constantly during the whole life of the pod, this is intended to allow kubelet to know when to destroy a pod and create a new one, when defining a liveness probe keep in mind it should return a success code constantly while your application is running, do not configure it for something that is only "true" while starting or a short period of time as that will cause kubelet to kill your pod when that test if failed but your application might be totally healty.
Readiness probes
This probe is probably the most important one for the topic we are covering (knowing when your application can accept traffic). The readiness probe will be executed only once just after your startup probe (if defined) and once it completes it will let Kubernetes determine what to do with your pod. If the probe is a success the pod IP will be added in the backed of the apropriate service to start receiving traffic, if the probe is failed Kubrenetes will kill your pod and start a new one (which will do all the probes defined again), this will prevent an unhealty pod to receive requests.
Wrapping up
So, basically our deployment should specify the three probes, the "startup" will fire as soon as the pod is scheduled into a node, upon completion a "readiness" probe will be fires which can return a success code which will add the IP of the pod into the service backend but if it fails kubelet will kill the pod and restart the process again. If the readiness probe completes then your application is healthy and will start receiving traffic but then during it's whole life kubelet will keep executing periodically a "liveness" probe on the pod to determine if it is needed to kill the pod or not, on the first failure from the liveness probe, your pod will be restarted.
Reference material
This is a short article to summarize the concept of probes in Kubernetes, there are a lot of technical details on how to properly implement and configure them, if you need more details on it, you can find it in the official documentation here.
Top comments (0)