API container crashes often when generating nuxt pdf
For a long time now, we have been getting lots of sentry errors about unavailable endpoints during nuxt print. On prod this has not been a huge problem, the application has mostly remained up, as our uptime stats show. This is probably due to our multiple replicas and Kubernetes' auto-scaling behaviour, and because our API container starts relatively quickly.
However, when working on print topics locally, the same may occasionally happen. I noticed it often when live updating something on the print configurator, and the nuxt print preview is triggered. It happens maybe 1 out of 10 or 20 times that the API container crashes during that process.
We could simply set restart: always in docker-compose.yml as a band-aid fix. However, maybe it would be better to find the root cause and fix that.
I would vote to not use restart: always, this leads to container names beeing used when you forget to stop the docker compose setup.
Better options are:
on-failure[:max-retries]: The policy restarts the container if the exit code indicates an error. Optionally, limit the number of restart retries the Docker daemon attempts.
unless-stopped
https://docs.docker.com/reference/compose-file/services/#restart
And best: fix the underlying issues.
We already had that problem with the e2e tests:
- https://github.com/ecamp/ecamp3/issues/5493#issuecomment-2240057818
- https://github.com/ecamp/ecamp3/pull/5560
This is not a permanent solution for development. (Except when you don't change anything in the api, you can use the prod api image)
I have realized the problem on my side may be that on changing the branch (or even when changing something in the API), API Platform rebuilds its cache and the OpenAPI docs, and runs into an out of memory error in the process. I think the OOM error I get says something about roughly 137MB of memory, which seems quite low to me. On the frankenphp documentation they say frankenphp does not come with any default php.ini.
So far it seems to help if I add the following line to the frankenphp block in api/docker/Caddyfile and rebuilding the api container:
php_ini memory_limit 500M
Obviously the memory_limit should be set in some other place so that it only affects development and not production.