charts icon indicating copy to clipboard operation
charts copied to clipboard

[mealie] api container missing health checks, causes service to go ready before it can serve traffic

Open disconn3ct opened this issue 1 year ago • 3 comments

Helm chart name

mealie

Helm chart version

5.0.2

Container name

hkotel/mealie

Container tag

api-v1.0.0beta-3

Description

The default values have health checks for the frontend container, but not the api. This results in the pod going ready as soon as the frontend container is up, but before the API container is ready. The browser is shown:

Server error
An error occurred in the application and your page could not be served. If you are the application owner, check your logs for details.

Expected result

Pod stays unready until all containers are able to service requests

Helm values to reproduce

(Default)

Additional Information

Found logs, pretty expected:

mealie/mealie-beta-6dc767dd9-2cw7f[mealie-beta]:  ERROR  connect ECONNREFUSED 127.0.0.1:9000
mealie/mealie-beta-6dc767dd9-2cw7f[mealie-beta]: 
mealie/mealie-beta-6dc767dd9-2cw7f[mealie-beta]:   at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1187:16)

Repo link

No response

disconn3ct avatar Jul 17 '22 14:07 disconn3ct

By default our charts only check if a single port is able to connect, in this case the frontend. Ideally the frontend wouldn't report 'ready' if the api is not available. We could maybe do this with a http check, or worst case some kind of shell probe

bjw-s avatar Jul 17 '22 17:07 bjw-s

If the api container has a problem, it will never be restarted. The cluster will just cycle the (otherwise working) frontend container forever. That is why health checks are applied to individual containers, not a single check for the whole pod.

disconn3ct avatar Jul 18 '22 13:07 disconn3ct

You are right, I somehow had it in my mind that the probes are per Pod 🤦

bjw-s avatar Jul 18 '22 14:07 bjw-s