Add prometheus metrics

[varac]

Please add prometheus metrics so the prometheus can check for outdated containers and alertmanager can handle notifications. Similar ratinale as the related watchtower request.

[philosowaffle]

I know nothing about Go, but a bit about instrumenting with prometheus, thought I might help flesh out this idea a bit more.

This could be implemented as a new notif type in diun.

1. Go library for Prometheus
2. Docs on instrumenting Go

When publishing notifications, the prometheus notification handler would do something akin to:

1. Create a new gauge: diun_container_status
2. The Gauge at a minimum should expose the following labels: diun_container_status{container="the_container_name"}
3. For each container, if an update is available, set the Gauge to 1, else set the Gauge to 0

There is a ton of other stuff you can instrument using Prometheus metrics that exposes a lot of fun data. But at a minimum this would be viable for us to consume this data in Grafana where we already have the ability to fine tune and configure alerts.

The alert fine tuning that already exists in Grafana Alert Manager supports some of the features requested in issues #10 #109 #322

[crazy-max]

I will work soon to add support for Prometheus and maybe remote write with OpenTelemetry.

[laurivosandi]

Any updates on this? Also please export Docker image timestamp of the running images to easily determine if Docker image is not being updated by the author. This would also help identifying images which are not using latest tag and need manual updates

From microservice point of view you could just drop all the notification machinery from Diun and instead solely focus on exporting nice Prom metrics. Notifications should be handled and deduplicated by Alertmanager.

[AthrogatePEQ]

I have created a /v1/metrics api end point for prometheus style metrics.

I wrote my code based on an older version of diun. There's one change I still have to reconcile in the provider/kubernetes/pod.go I'd modified ValidateImage to work with an ImageDigest of the Status/ContainerStatus model.

So I just need to work those changes into the current master. Change can be seen at https://github.com/AthrogatePEQ/diun I'll work on a pull request as soon as I get the changes working. (it did work with a previous checkout of the master branch :) )

[moritzj29]

PR has been opened in the meantime: #754 Thanks @AthrogatePEQ ! Looking forward to this!

[jbouchery]

Hi !

I see that the https://github.com/crazy-max/diun/pull/754 PR is closed. Any update on this ?

I will try to continue the work of @AthrogatePEQ in the meantime.

[AthrogatePEQ]

I have opened a new pull request #775 (I closed the previous ones because I found a problem with the code and I didn't know how to fix it in a PR or prevent the PR from being consumed before I'd fixed the issues that I'd found.)

[VolanTom]

Hello there!

First a big THANKS to @crazy-max for Diun and @AthrogatePEQ for the metrics api!

Any idea when the pull request https://github.com/crazy-max/diun/pull/775 of @AthrogatePEQ will be review and merge?