Authenticating Grafana instances for Unified Alerting

[cyberkov]

Hello!

We would very much like to use Grafana's Unified Alerting feature for Loki and Mimir. Therefore it would be necessary to extend current authentication functionality to allow the use of a predefined key (signed JWT Token) as alternative to the oauth token sent by the User.

Grafana sends the following requests when querying for data for the alerting: Loki:

{"level":"debug","path":"/loki/api/v1/query_range","time":1729852030,"message":"Request complete"}

{"level":"debug","verb":"GET","request":"{\"method\":\"GET\",\"url\":\"/loki/api/v1/query_range?direction=backward\\u0026end=1729852030000000000\\u0026query=sum%28count_over_time%28%7Bhost_name%3D%22example.com%22%7D+%7C%3D+%60%60+%5B1s%5D%29%29\\u0026start=1729851430000000000\\u0026step=1000ms\",
 \"header\":{\"Accept-Encoding\":[\"gzip\"],
 \"Fromalert\":[\"true\"],
 \"User-Agent\":[\"Grafana/11.2.2\"],
 \"X-Loki-Response-Encoding-Flags\":[\"categorize-labels\"],
 \"X-Rule-Folder\":[\"XXXX\"],
 \"X-Rule-Name\":[\"XXX-testalert\"],
 \"X-Rule-Source\":[\"scheduler\"],
 \"X-Rule-Type\":[\"alerting\"],
 \"X-Rule-Uid\":[\"ce1xgwa9oi3uod\"],
 \"X-Rule-Version\":[\"2\"]},
 \"body\":\"[REDACTED]\"}",
 "path":"/loki/api/v1/query_range","time":1729852030}

Prometheus:

{"level":"debug","path":"/api/v1/query","time":1729852515,"message":"Request complete"}

{"level":"debug","verb":"POST","request":"
 {\"method\":\"POST\",
 \"url\":\"/api/v1/query\",
 \"header\":{
 \"Accept-Encoding\":[\"gzip\"],
 \"Content-Length\":[\"155\"],
 \"Content-Type\":[\"application/x-www-form-urlencoded\"],
 \"Fromalert\":[\"true\"],
 \"User-Agent\":[\"Grafana/11.2.2\"],
 \"X-Rule-Folder\":[\"XXX\"],
 \"X-Rule-Name\":[\"ApacheWorkersLoad\"],
 \"X-Rule-Source\":[\"scheduler\"],
 \"X-Rule-Type\":[\"alerting\"],
 \"X-Rule-Uid\":[\"ce1xhq06gtgcgf\"],
 \"X-Rule-Version\":[\"1\"]},
 \"body\":\"[REDACTED]\"}",
 "path":"/api/v1/query","time":1729852515}

According to https://github.com/grafana/grafana/blob/main/pkg/services/ngalert/models/constants.go it ought to be possible to use FromAlert=true as identification that the request is originating from the Alerting.

I would suggest to implement the signed jwt token containing the identifying user email e.g.

{
  "email": "[email protected]"
}

in a custom HTTP-Header like X-Multena-TokenAuth. It would then be possible to map that user to the underlying authorization layer (in my case the database).

I don't have a preference regarding on how to validate the signer. Probably ED25519 keys would make sense. in that case there would need to be a setting to provide the public key for validation.

As the token can be crafted to our liking, it still would be possible to craft a token, containing the admingroup to bypass the authorization process.

[cyberkov]

Hi!

I created a "workflow diagram" of how i think it might be useful to implement: