Let’s say we have an application inside docker image that connects to Apache Kafka on start and tries to publish a bunch of messages to topics. Now let’s imagine that you want to test your app locally and thus you’ve created a docker-compose.yml with Apache Kafka, Zookeeper, and your application image. Looks nice.
You are also didn’t forget to provide all necessary depends_on blocks and finally, the container load hierarchy looks like Zookeeper -> Kafka -> Your Application. Seems everything is correct but are you sure? In most cases, you will face the story when your application is unable to obtain a connection to a Kafka broker since it is not yet ready. depends_on checks only the fact that the container is started, but doesn’t guarantee anything related to its inner functionality and state.
What is healthcheck?
Healthcheck is an action that checks a specific piece of functionality inside a running container. If a healthcheck returns success that means that the invoked function was successful and therefore the container works as expected and that it is ready to go. Otherwise, docker will try to invoke failed healthchecks again and again according to the strategy given in docker-compose.yml. In the worst-case scenario, the whole docker-compose will fail to start since some healthchecks weren’t successful.
Creating healthchecks for Apache Kafka
The simplest way to create a healthcheck for Kafka container is to run netcat against the localhost:9092:
kafka:
image: confluentinc/cp-kafka
depends_on:
- zookeeper
healthcheck:
test: nc -z localhost 9092 || exit -1
start_period: 15s
interval: 5s
timeout: 10s
retries: 10
The start_period provides initialization time for containers that need time to bootstrap. Probe failure during that period will not be counted towards the maximum number of retries. However, if a health check succeeds during the start period, the container is considered started and all consecutive failures will be counted towards the maximum number of retries (see here). Other parameters seem to be self-explanatory.
Settings for dependent conatainers
To make any container dependent on a healthcheck status of another container add the following lines to the docker-compose.yml:
depends_on:
kafka:
condition: service_healthy
Conclusion
By using healthchecks in docker you’re able to control an inner container state before actually starting any connections and jobs around it. That forces your environment to be in a consistent state and makes the execution much more predictible.
