immich
immich copied to clipboard
[Bug]: Handle SIGTERM
The node processes started by Immich currently don't handle the SIGTERM signal. This is required to let Docker gracefully shutdown the containers.
Ideally this should close any open connections and also be logged.
Affected:
- [ ] machine-learning
- [ ] microservices
- [ ] server
- [ ] web
Just a comment to give this issue a bit more attention. Especially for my Podman installation where I run containers using systemd, I have to hack a bit to manage the units.
Maybe this is a solution: https://stackoverflow.com/questions/43584760/gunicorn-graceful-stopping-with-docker-compose (for ML)
For the services, catching the signals in the script might work?
I'm just thinking out loud here. I'm a mere beginner when it comes to containers.
The container itself should already be setup correctly. It's just the node logic which is missing here.
You mean the containers itself stop with SIGTERM but the business logic isn't gracefully aborted?
Then I have a bug in my setup because some of my containers still fallback to SIGKILL after the standard 10s timeout.
I should phrase it differently. The signal from the docker daemon is correctly handed to the node process which ignores it at the moment. The daemon ultimately waits 10s until it kills the processes in a non-graceful manner using SIGKILL.
tl;dr immich processes need to handle SIGTERM
https://github.com/nodejs/docker-node/blob/main/docs/BestPractices.md#handling-kernel-signals
Looks like we need to add an init process to our docker containers. Maybe we can get away with a simple change to our compose file?
https://docs.docker.com/compose/compose-file/05-services/#init