HTTP/2 not working for `grafana_synthetic_monitoring_check`

[justinTM]

Terraform Version

• Terraform: 1.0.0
• Terraform Grafana Provider: v1.16.0
• Grafana: v8.5.4 (78ee95814)

Affected Resource(s)

Please list the resources as a list, for example:

• grafana_synthetic_monitoring_check

If this issue appears to affect multiple resources, it may be an issue with Terraform's core, so please mention this.

Terraform Configuration Files

# config.yml
# default values to apply for each check
check_defaults:
  probes:
    - Atlanta
    - Dallas
    - NorthVirginia
    - Oregon
    # - Amsterdam
    # - Atlanta
    # - Bangalore
    # - CapeTown
    # - Chicago
    # - Frankfurt
    # - London
    # - LosAngeles
    # - Miami
    # - Mumbai
    # - NewYork
    # - Newark
    # - NorthCalifornia
    # - NorthVirginia
    # - Ohio
    # - Paris
    # - SanFrancisco
    # - SanJose
    # - SaoPaulo
    # - Seattle
    # - Seol
    # - Seoul
    # - Singapore
    # - Sydney
    # - Tokyo
    # - Toronto
  enabled: true
  allowed_types:
    - http
    - ping
    - dns
    - tcp
  # Settings for HTTP check. The target must be a URL (http or https).
  type: http
  http:
    # Token for use with bearer authorization header.
    bearer_token: ""
    # The body of the HTTP request used in probe.
    body: ""
    # The name of the query parameter used to prevent the server from using a cached response. Each probe will assign a random value to this parameter each time a request is made.
    cache_busting_query_param_name: ""
    # List of regexes. If any match the response body, the check will fail.
    fail_if_body_matches_regexp: []
    # List of regexes. If any do not match the response body, the check will fail.
    fail_if_body_not_matches_regexp: []
    # Fail if SSL is present. Defaults to false.
    fail_if_ssl: false
    # Fail if SSL is not present. Defaults to false.
    fail_if_not_ssl: false
    # The HTTP headers set for the probe
    headers: []
    # Options are V4, V6, Any. Specifies whether the corresponding check will be performed using IPv4 or IPv6. The Any value indicates that IPv6 should be used, falling back to IPv4 if that's not available. Defaults to V4.
    ip_version: V4
    # Request method. One of GET, CONNECT, DELETE, HEAD, OPTIONS, POST, PUT, TRACE Defaults to GET.
    method: GET
    # Do not follow redirects. Defaults to false.
    no_follow_redirects: false
    proxy_url: ""
    # List of valid HTTP versions. Options include HTTP/1.0, HTTP/1.1, HTTP/2
    valid_http_versions:
      - HTTP/1.0
      - HTTP/1.1
      - HTTP/2
    # Accepted status codes. If unset, defaults to 2xx.
    valid_status_codes:
      - 200
      - 201
      - 202
      - 203
      - 204
      - 205
      - 206
      - 207
      - 208
      - 226
    tls_config: {
      # ca_cert (String) CA certificate in PEM format.
      # client_cert (String) Client certificate in PEM format.
      # client_key (String, Sensitive) Client key in PEM format.
      # insecure_skip_verify (Boolean) Disable target certificate validation. Defaults to false.
      # server_name (String) Used to verify the hostname for the targets.
    }
    basic_auth: {
      # password (String) Basic auth password.
      # username (String) Basic auth username.
    }
    # Check fails if headers match.
    fail_if_header_matches_regexp: {
      # header (String, required) Header name.
      # regexp (String, required) Regex that header value should match.
      # Allow header to be missing from responses. Defaults to false.
    }
    # Check fails if headers do not match.
    fail_if_header_not_matches_regexp: {
      # header (String, required) Header name.
      # regexp (String, required) Regex that header value should match.
      # Allow header to be missing from responses. Defaults to false.
    }

# checks.tf

# make one Synthetic Monitoring check for each "checks" list item in config.yml
resource "grafana_synthetic_monitoring_check" "all" {
  for_each = local.checks_clean_for_output

  job     = each.value["job"]
  target  = each.value["target"]
  enabled = each.value["enabled"]
  labels  = merge(local.global_tags, lookup(each.value, "labels", {}))

  # list of probe numbers from city names in defaults combined with per-check probe city names
  probes  = [ for city in each.value.probes : lookup(local.probes_map, city) ]

  settings {
    # allow each check to override default settings for checks of type "http"
    dynamic "http" {
      for_each = each.value.type == "http" ? [each.value.http] : []

      content {
        bearer_token                   = http.value.bearer_token
        body                           = http.value.body
        cache_busting_query_param_name = http.value.cache_busting_query_param_name
        fail_if_body_matches_regexp    = http.value.fail_if_body_matches_regexp
        fail_if_not_ssl                = http.value.fail_if_not_ssl
        fail_if_ssl                    = http.value.fail_if_ssl
        headers                        = http.value.headers
        ip_version                     = http.value.ip_version
        method                         = http.value.method
        no_follow_redirects            = http.value.no_follow_redirects
        proxy_url                      = http.value.proxy_url
        valid_http_versions            = http.value.valid_http_versions
        valid_status_codes             = http.value.valid_status_codes

        dynamic "tls_config" {
          for_each = http.value.tls_config
          content {
            server_name = tls_config.value.server_name
            client_cert = tls_config.value.client_cert
          }
        }
        dynamic "basic_auth" {
          for_each = http.value.basic_auth
          content {
            username = basic_auth.value.username
            password = basic_auth.value.password
          }
        }
        dynamic "fail_if_header_matches_regexp" {
          for_each = http.value.fail_if_header_matches_regexp
          content {
            header        = fail_if_header_matches_regexp.value.header
            regexp        = fail_if_header_matches_regexp.value.regexp
            allow_missing = fail_if_header_matches_regexp.value.allow_missing
          }
        }
      }
    }
  }
}

Debug Output

In Grafana Checks dashboard logs:

label_ManagedBy=Terraform msg="Invalid HTTP version number" version=HTTP/2.0

Panic Output

If Terraform produced a panic, please provide a link to a GitHub Gist containing the output of the crash.log.

Expected Behavior

HTTP/2 should be an accepted option. The documentation says:

valid_http_versions = [
        "HTTP/2",
      ]

... which is what we are using.

Actual Behavior

The check fails with a message about an invalid HTTP version number.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply

Important Factoids

Are there anything atypical about your accounts that we should know? For example: Running in EC2 Classic? Custom version of OpenStack? Tight ACLs?

References

Are there any other GitHub issues (open or closed) or Pull Requests that should be linked here? For example:

• GH-1234

[inkel]

@justinTM thanks for the report. Could this be tied to this Synthetic Monitoring incident? Are you still experiencing this issue?

[justinTM]

hey @inkel it's been doing this for a few months but it wasn't an issue for us until now. I tried using "HTTP/2.0" as well as "HTTP/2" and neither work.

[mem]

Question: what is failing is the actual check when it runs? Not during terraform application? (meaning, the check gets added correctly)

[mem]

Sorry, it took me some squinting to notice: can you try with "HTTP/2.0"?

It's a quirk of the way the underlying code works. I think it's seeing "HTTP/2" in the response and internally translating that to "HTTP/2.0"

[justinTM]

@mem actually yeah, this is the wrong place to open the bug report; terraform applies the check successfully but the check itself fails.

Let me try with HTTP/2.0 and report back.