[FR] Add authorization with an API KEY token

[qmonitoring]

Use case: I am trying to scrape an endpoint with an API KEY аuthorization using Prometheus with json-exporter ( prom/prometheus:main and prometheuscommunity/json-exporter:v0.3.0 docker images):

docker run -d --network=host --name=prometheus -v /path/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus:main --config.file=/etc/prometheus/prometheus.yml --web.listen-address=:9095

docker run -d --name=json-exporter --network=host -v /path/jsonexporter/config.yml:/config.yml prometheuscommunity/json-exporter:v0.3.0 --config.file=/config.yml

with the following Prometheus prometheus.yml configuration file:

global:
  evaluation_interval: 15s
  scrape_interval: 15s
  scrape_timeout: 10s
  external_labels:
    environment: prometheus_in_docker
	
scrape_configs:

  - job_name: json_exporter_self
    static_configs:
      - targets:
        - <host>:7979 ## Location of the json exporter's real <hostname>:<port>

  ## gather the metrics from third party json sources, via the json exporter
  - job_name: json_exporter_data
    metrics_path: /probe
    authorization:
      credentials: '{my_secret_token}'
      type: apikey	
    static_configs:
      - targets:
        - http://<endpoint_host>:<port>/api/v1/data
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: <host>:7979 ## Location of the json exporter's real <hostname>:<port>

and the following json-exporter config.yml configuration file:

metrics:
- name: example_metric
  path: "{ .data }"
  help: Example 
  labels:
    environment: testdata # static label

prom/prometheus:main and prometheuscommunity/json-exporter:v0.3.0 are up and running, self scraping works fine (http://:7979/metrics), but a json scraping doesn't work (http://:7979/probe), the error is the following:

Failed to fetch JSON response. TARGET: http://<endpoint_host>:/api/v1/data, ERROR: 401 Unauthorized

The endpoint itself is available: curl -X GET 'http://<endpoint_host>:/api/v1/data' -H 'Authorization: {my_secret_token}'

Feature request: Could you please implement authorization with API KEY token?

(please see link with additional discussion in the Prometheus repository)

[busa-blade]

Does anyone know if this has been implemented?

[alinix-alinix]

Does anyone know if this has been implemented?

I have a similar issue. Trying to scrape json path via api url like http://MyHost/platform_api/GetSomeStuff/?account_id=123456&api_key=11112222233334444 It returns json in browser or curl just fine. But i have no success probing that url with json_exporter http://192.168.9.103:7979/probe?target=http://MyHost/platform_api/GetSomeStuff/?account_id=123456&api_key=11112222233334444 It seems to pass an account id but no api_key, so i'm getting authorization error.

[dahoat]

I have a similar issue. If you run json_exporter locally and use Wireshark, you can see that the exporter only forwards the first get argument.

[dahoat]

Sorry I didn't wait with my last comment until I knew the solution:

Solution

Replace & with %26 because you have to escape your &.

Root cause

If you request http://exporter.local/probe?target=someurl&apiKey=abcd then apiKey is delivered as GET parameter to probe and is not read as part of the string intended for target.
Therefore, request http://exporter.local/probe?target=someurl%26apiKey=abcd

Code location

https://github.com/prometheus-community/json_exporter/blob/d1456b89177b93ed15ef6fc43dc0b4c529599c3c/cmd/main.go#L110

[acondrat]

I think this is already supported with the http_client_config. Look at the example provided in https://github.com/prometheus-community/json_exporter/blob/master/examples/config.yml

I needed something similar in a Kubernetes environment and my config looks like

          http_client_config:
            authorization:
              type: Key
              credentials_file: /tmp/api-key.txt

Translated to curl this would be something like

curl -H "Authorization: Key MySecretApiKey" "exporter:7979/probe?target=http://mytarget"

For other options see https://pkg.go.dev/github.com/prometheus/common/config#Authorization

[urpylka]

We need to have ability to add custom headers and hide it. For example gitlab api provide to use header field - token to authentificate. Yes, maybe gitlab team made bad solution, but, for now, I don't ability to use secure.

[morremeyer]

Authorization with the standardized HTTP Authorization methods is possible as @acondrat described in https://github.com/prometheus-community/json_exporter/issues/87#issuecomment-1305469450.

For example, the following module configuration sets the header Authorization: Bearer 123456 when the file /credentials/some_app.txt contains the string 123456.

modules:
  some_application:
    http_client_config:
      authorization:
        credentials_file: /credentials/some_app.txt

I don't think json_exporter should support non-standard-conformant authentication methods since this would add maintenance overhead that is not rectifiable - any application should support at least one of the standardized methods of Bearer tokens, Basic auth, or OAuth2.

[schoen2]

Hi @morremeyer,

I hope you can help me out. I have deployed Json-Exporter and Prometheus as a Helm chart in my Kubernetes cluster.

I have an API query that I can successfully submit in the terminal and get a return:

curl -X GET https://api.viessmann.com/iot/v2/features/installations/ID/gateways/ID/devices/ID/features -H "Authorization: Bearer <token>“

If I go by the http_client_config documentation (https://pkg.go.dev/github.com/prometheus/common/config#HTTPClientConfig) bearer_token or authorization.credentials must be of type Secret. Unfortunately, I can't get the bearer token to work properly as a Kubernetes secret.

This is what my json exporter config looks like:

configuration:
  config: |
    ---
    modules:
      some_application:
        metrics:
        - name: viessmann
          type: object
          path: '{ .data.0 }'
          help: viessmann
          labels:
            id: "viessmann"
            name: "viessmann"
          values:
            apiVersion: '{.apiVersion}'
        http_client_config:
          bearer_token: 'viessmann-secret'
or
        http_client_config:
          authorization:
            credentials: 'viessmann-secret'

The Kubernetes-secret I create like:

apiVersion: v1
kind: Secret
metadata:
  name: viessmann-secret
  namespace: monitoring
type: Opaque
data:
  bearer_token: BASE64_ENCODED_TOKEN_VALUE_HERE

I hope you or someone else can help me :/ Thanks in advance

[morremeyer]

Hey @schoen2, let me post the full configuration that we use.

The http_client_config.bearer_token or http_client_config.authorization.credentials need to be the literal token, not a reference to a secret with the token.

If you want to read the token from a secret, which I would recommend, you'll need to mount the secret as a file and reference that with http_client_config.authorization.credentials_file.

To illustrate, let me add the config we have deployed currently.

Deploying with helm for renovate-ce

We're using https://github.com/prometheus-community/helm-charts/tree/main/charts/prometheus-json-exporter in version 0.11.0 to deploy.

The values we have configured in the values file

serviceMonitor:
  enabled: true
  targets:
    - name: renovate-ce
      url: http://the-endpoint-for-renovate
      module: renovate

configuration:
  config: |
    ---
    modules:
      renovate:
        http_client_config:
          authorization:
            credentials_file: /credentials/renovate/renovate-ce.txt

        metrics:
          - name: renovate_organization_count
            valuetype: gauge
            path: "{ .app.organizationCount }"
            help: The number of organizations known to renovate

          - name: renovate_repository_count
            valuetype: gauge
            path: "{ .app.repositoryCount }"
            help: The number of repositories known to renovate

          - name: renovate_jobs_processed
            valuetype: counter
            path: "{ .jobs.history.processed }"
            help: The number of jobs processed since the instance has started

          - name: renovate_queue_size
            valuetype: gauge
            path: "{ .jobs.queue.size }"
            help: The number of jobs currently in the queue

additionalVolumes:
  - name: credentials-renovate-ce
    secret:
      secretName: credentials-renovate-ce

additionalVolumeMounts:
  - name: credentials-renovate-ce
    mountPath: /credentials/renovate/renovate-ce.txt
    subPath: mendRnvServerApiSecret

The secret

apiVersion: v1
kind: Secret
metadata:
  name: credentials-renovate-ce
  namespace: kube-prometheus-stack
type: Opaque
data:
  mendRnvServerApiSecret: Tm8gY2hhbmNlLCBJIHJlcGxhY2VkIHRoaXMgYmVmb3JlIHBhc3RpbmcgaXQ=

[schoen2]

Great!! Thanks a lot! I don't know the difference between credentials_file and credentials about the reference to a secret.