HTTP Probe Failed when request URL is not encoded

[wailek91]

blackbox_exporter version: output of blackbox_exporter --version

0.20.0

What is the blackbox.yml module config.

    modules:
      http_2xx:
        prober: http
        timeout: 50s
        http:
          valid_http_versions: ["HTTP/1.1", "HTTP/2.0"]
          valid_status_codes: [200, 301, 302]  # Defaults to 2xx
          method: GET
          follow_redirects: true
          fail_if_ssl: false
          fail_if_not_ssl: true
          preferred_ip_protocol: "ip4"
          headers:
            Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
            Accept-Encoding: gzip, deflate, br
            User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36
            Accept-Language: \*
      http_post_2xx:
        prober: http
        http:
          method: POST 

What is the prometheus.yml scrape config.

- job_name: 'blackbox'
      metrics_path: /probe
      scrape_interval: 1m30s
      scrape_timeout: 1m
      params:
        module: [http_2xx]  # Look for a HTTP 200 response.
      file_sd_configs:
      - files:
        - /targets/blackbox_targets.yaml
        refresh_interval: 1m
      relabel_configs:
        - source_labels: [__address__]
          target_label: __param_target
        - source_labels: [__param_target]
          target_label: instance
        - target_label: __address__
          replacement: 127.0.0.1:9115  # The blackbox exporter's real hostname:port.

What logging output did you get from adding &debug=true to the probe URL?

Logs for the probe:
ts=2022-04-08T07:21:13.555333258Z caller=main.go:347 module=http_2xx target=https://tmrw-th.onelink.me/zL1L/dc78396d level=info msg="Beginning probe" probe=http timeout_seconds=50
ts=2022-04-08T07:21:13.555448253Z caller=http.go:334 module=http_2xx target=https://tmrw-th.onelink.me/zL1L/dc78396d level=info msg="Resolving target address" target=tmrw-th.onelink.me ip_protocol=ip4
ts=2022-04-08T07:21:13.579076361Z caller=http.go:334 module=http_2xx target=https://tmrw-th.onelink.me/zL1L/dc78396d level=info msg="Resolved target address" target=tmrw-th.onelink.me ip=13.33.96.23
ts=2022-04-08T07:21:13.57920286Z caller=client.go:252 module=http_2xx target=https://tmrw-th.onelink.me/zL1L/dc78396d level=info msg="Making HTTP request" url=https://13.33.96.23/zL1L/dc78396d host=tmrw-th.onelink.me
ts=2022-04-08T07:21:14.306507818Z caller=client.go:498 module=http_2xx target=https://tmrw-th.onelink.me/zL1L/dc78396d level=info msg="Received redirect" location="https://www.tmrwbyuob.com/th/en/partnership_big.html?shortlink=dc78396d&pid=Partnership&c=BIG Rewards (AABGTMRW0621) EN&af_channel=Partnership"
ts=2022-04-08T07:21:14.306548386Z caller=client.go:252 module=http_2xx target=https://tmrw-th.onelink.me/zL1L/dc78396d level=info msg="Making HTTP request" url="https://www.tmrwbyuob.com/th/en/partnership_big.html?shortlink=dc78396d&pid=Partnership&c=BIG Rewards (AABGTMRW0621) EN&af_channel=Partnership" host=
ts=2022-04-08T07:21:14.30656013Z caller=client.go:252 module=http_2xx target=https://tmrw-th.onelink.me/zL1L/dc78396d level=info msg="Address does not match first address, not sending TLS ServerName" first=13.33.96.23 address=www.tmrwbyuob.com
ts=2022-04-08T07:21:14.364225215Z caller=main.go:140 module=http_2xx target=https://tmrw-th.onelink.me/zL1L/dc78396d level=info msg="Received HTTP response" status_code=400
ts=2022-04-08T07:21:14.364258474Z caller=main.go:140 module=http_2xx target=https://tmrw-th.onelink.me/zL1L/dc78396d level=info msg="Invalid HTTP response status code" status_code=400 valid_status_codes="[200 301 302]"
ts=2022-04-08T07:21:14.364286015Z caller=main.go:140 module=http_2xx target=https://tmrw-th.onelink.me/zL1L/dc78396d level=info msg="Response timings for roundtrip" roundtrip=0 start=2022-04-08T07:21:13.579309726Z dnsDone=2022-04-08T07:21:13.579309726Z connectDone=2022-04-08T07:21:13.580891463Z gotConn=2022-04-08T07:21:13.585831327Z responseStart=2022-04-08T07:21:14.306403072Z tlsStart=2022-04-08T07:21:13.580928705Z tlsDone=2022-04-08T07:21:13.585754918Z end=0001-01-01T00:00:00Z
ts=2022-04-08T07:21:14.364317792Z caller=main.go:140 module=http_2xx target=https://tmrw-th.onelink.me/zL1L/dc78396d level=info msg="Response timings for roundtrip" roundtrip=1 start=2022-04-08T07:21:14.306639488Z dnsDone=2022-04-08T07:21:14.354433793Z connectDone=2022-04-08T07:21:14.356427082Z gotConn=2022-04-08T07:21:14.361902217Z responseStart=2022-04-08T07:21:14.364138333Z tlsStart=2022-04-08T07:21:14.356478478Z tlsDone=2022-04-08T07:21:14.361821722Z end=2022-04-08T07:21:14.364275592Z
ts=2022-04-08T07:21:14.364389207Z caller=main.go:347 module=http_2xx target=https://tmrw-th.onelink.me/zL1L/dc78396d level=error msg="Probe failed" duration_seconds=0.809020933

What did you do that produced an error?

URL / Redirected URL containing non-encoded whitespace. For example, "www.example.com/test?key1=value 1".

What did you expect to see?

HTTP Probe success.

What did you see instead?

HTTP 400 returned.

[electron0zero]

can you try setting compression in check config, see: https://github.com/prometheus/blackbox_exporter/blob/master/CONFIGURATION.md#http_probe

[wailek91]

unfortunately the issue is not related to compression setting. I tried with compression: gzip and it still failed with HTTP 400.

I did tried to replace encoded query to the targetUrl.RawQuery. The HTTP probe will be sending the correct HTTP request URL to the server and respond with HTTP 200.

[mem]

targets/blackbox_targets.yml contains www.example.com/test?key1=value 1?

I'm not quite sure why that should work. Can you provide a use case?

[mmpetarpeshev]

I have the same issue, even without the special url redirect . Simple target url is failing with 404 , it works with curl or browsers.