keycloak-metrics-spi

keycloak-metrics-spi copied to clipboard

Description

Add authentication to the metrics endpoint. It should be only accessible with an specific user role oder admin account.

Expected Behavior

http://127.0.0.1:8080/auth/realms/master/metrics should be return 503 unless a valid session with specific role logged in.

Actual Behavior

http://127.0.0.1:8080/auth/realms/master/metrics is would accessible.

@jkroepke having a way to restrict access to the metrics endpoint sounds reasonable. Usually this endpoint is scraped by Prometheus which won't have a valid keycloak session. Wondering what's the best way here, i'll try to give it some thought.

Some inspiration from here https://github.com/AndreyVMarkelov/jira-prometheus-exporter/

https://github.com/AndreyVMarkelov/jira-prometheus-exporter/blob/master/src/main/java/ru/andreymarkelov/atlas/plugins/promjiraexporter/servlet/PrometheusExporter.java#L35

This exporter generates a static key. If you want to scrape the endpoint you must attach the key as get parameter. Otherwise you get an unauthorized messages.

2 possible ways to store the static key inside keycloak.

• The User must be define the key via -D property on startup
• The realm object (https://www.keycloak.org/docs-api/5.0/rest-api/index.html#_realmrepresentation) supports a attribute attributes which is designed for user properties. Maybe it's possible to safe the key there.

I am interested in this. Having those keycloak metrics open to everyone does not seem like good idea.

@jkroepke did you managed to configure it? Thanks

As a workaround and since I have an HAproxy in front of my Keycloak servers, I added a http-request deny to my HAproxy frontend configuration:

frontend https-keycloak
  ...
  balance  roundrobin
  http-request  deny if { path -i -m reg /auth/realms/.+\/metrics } !{ src <IP_PROMETHEUS_SERVER> }
  mode  http

This config seems to do the trick.

One way to restrict metrics endpoint is mentioned under Keycloak admin docs https://www.keycloak.org/docs/11.0/server_admin/#admin-endpoints-and-console where you replace path-prefix('/auth/admin') by either path-template('/auth/realms/{realm}/metrics') or (path-prefix('/auth/admin') or path-template('/auth/realms/{realm}/metrics'))

I've fixed this problem with an nginx location rule and basic auth (ofcourse servering on https)

    location ~ /auth/realms/[a-z]+/metrics  {
        auth_basic "Prometheus exporter";
        auth_basic_user_file /etc/nginx/auth/prometheus;
        proxy_set_header    Host               $host;
        proxy_set_header    X-Real-IP          $remote_addr;
        proxy_set_header    X-Forwarded-For    $proxy_add_x_forwarded_for;
        proxy_set_header    X-Forwarded-Proto  $scheme;
        proxy_ssl_verify    off;
        proxy_pass https://localhost:8443;
    }

    location / {
        proxy_set_header    Host               $host;
        proxy_set_header    X-Real-IP          $remote_addr;
        proxy_set_header    X-Forwarded-For    $proxy_add_x_forwarded_for;
        proxy_set_header    X-Forwarded-Proto  $scheme;
        proxy_ssl_verify    off;
        proxy_pass https://localhost:8443;
    }

Prometheus itself supports base64 authentication.

I still would like to see an in-tree solution without the need to configure an reverse proxy.

Here is my NGINX Ingress Config to block requests to /auth/realms/master/metrics

   annotations: 
    kubernetes.io/ingress.class: "nginx"
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/server-snippet: |
      location /auth/realms/master/metrics {
          return 403;
      }

The location should probably be location ~ /auth/realms/[^/]*/metrics to get all realms.

I think it is really important to restrict this endpoint from regular users as well.

Seems like the best and easiest way is configuring it in the keycloak configuration.

As @giner mentioned:

One way to restrict metrics endpoint is mentioned under Keycloak admin docs https://www.keycloak.org/docs/11.0/server_admin/#admin-endpoints-and-console where you replace path-prefix('/auth/admin') by either path-template('/auth/realms/{realm}/metrics') or (path-prefix('/auth/admin') or path-template('/auth/realms/{realm}/metrics'))

Maybe just add the jboss cli command in the documentation, at least for now.

There is also a way to implement everything as and "admin rest api" but it requires some work, so I'm not sure if this is the approach you want to go with.

It seems that with KeyCloak 17 there is no way to rextrict endpoints anymore within the server configuration (https://github.com/keycloak/keycloak/issues/12394)

Maybe just add an option for some basic authentication. This way not the whole internet will be able to sniff out the metrics