grafana
Migrate Mimir to distroless Docker image
We should migrate Mimir to a distroless Docker image, to reduce the attack surface.
The main pushback towards this move was because it's then very difficult to debug a live container (e.g. look at files on disk or sockets), but in recent K8S versions we could use kubectl debug (doc) to add a debugging container to a running pod.
Nice, I didn't know about kubectl debug before. The debuggability would have been my only reason to not use a distroless image, so this reason is void now.
I like the idea, but kubectl debug and ephemeral containers are new feature in Kubernetes (marked as stable since 1.25, released in late august 2022), and not everyone has it available yet. For example it is not enabled in our dev GCP cluster. How long should we wait before proceeding with this?
How long should we wait before proceeding with this?
This issue originates from a conversation I had with Thomas (head security at Grafana) and Bryan.
My idea is to publish both alpine and distroless images for Mimir (and GEM) so that users / customers can choose. Moreover, we should also pubilsh a mimir-debug image to use for debugging purposes (with all our tooling inside, e.g. s3/gcs/azure clients). At Grafana we could slowly migrate to distroless, and share our learnings (documentation, improve the debug image, ...).
WDYT?
My idea is to publish both alpine and distroless images for Mimir (and GEM) so that users / customers can choose. Moreover, we should also pubilsh a
mimir-debugimage to use for debugging purposes (with all our tooling inside, e.g. s3/gcs/azure clients). At Grafana we could slowly migrate to distroless, and share our learnings (documentation, improve the debug image, ...).WDYT?
Publishing both sounds like a good idea. Internally we don't use Mimir docker images in any way, so that's a separate conversation.