rabbitmqctl in livenessProbe and readinessProbe keep RabbitMQ nodes at ~30% CPU when idle

[adamhooper]

Category:

Kubernetes apps

Type:

• [ ] Bug
• [X] Feature Request
• [ ] Process

---

Currently, RabbitMQ liveness checks are done using exec rabbitmqctl status. Launching rabbitmqctl costs 100% of a CPU for ~1.5s on an n1-standard-2 instance. That's because Erlang takes a long time to start up.

livenessProbe and readinessProbe each run every 10s. Each takes 1.5s to complete. Together, they cost 30% of all my pods' available CPU cycles.

There's an alternative: curl. The management API includes /api/aliveness-test/vhost which returns 200 {"status":"ok"} on success and 200 (yes, 200) {"status":"failed",...}. See https://rawcdn.githack.com/rabbitmq/rabbitmq-management/v3.7.8/priv/www/api/index.html

I propose adding curl (or some other HTTP-request command) to the RabbitMQ image and changing the livenessProbe and readinessProbe to use curl instead of rabbitmqctl. I expect RabbitMQ will be 30% more responsive as a result.