Adhoc tests utilize the wrong Loki datasource when looking for a response and timeout

[ckbedwell]

What happened: In certain situations* when you have multiple Loki datasources set-up in your instance, when utilizing the test button on the check form the UI says 'timed out waiting for a response'. This can be seen in Grafana Ops currently when utilizing the test button for any check.

*From some testing it seems to be when you have a Loki datasource with a uid of grafanacloud-logs which is not the datasource your SM instance is configured to use in your app provisioning.

What you expected to happen: That the correct Loki datasource is queried and we get the right logs instead of a false timeout.

How to reproduce it (as minimally and precisely as possible): In my local set-up I can repro this by provisioning an extra Loki instance and adding it to my datasource provisioning set-up:

- name: grafanacloud-ckbedwellksix-logs
    type: loki
    access: proxy
    url: https://logs-dev-005.grafana-dev.net
    basicAuth: true
    basicAuthUser:<user>
    secureJsonData:
      basicAuthPassword: <password>
    jsonData:
      maxLines: 1000
  - name: Loki
    uid: grafanacloud-logs
    type: loki
    url: https://logs-dev-005.grafana-dev.net
    basicAuth: true
    basicAuthUser:<user>
    isDefault: false
    secureJsonData:
      basicAuthPassword: <password>
    jsonData:
      derivedFields:
        - datasourceUid: grafanacloud-traces
          matcherRegex: '[tT]race_?[iI][dD]\"?[:=]\"?(\\w+)'
          name: traceID
          url: ${__value.raw}
      timeout: 300

Related Slack thread