NewRelic `no values found` error

[rp-martinbarmaymon]

Describe the bug

When querying NewRelic with canary MetricTemplates the query is being executed correctly and flagger logs the correct response JSON, however, Flagger expects a different response format from the one provided by NR. The structure expected is a key field 'result' that is then used to retrieve the query result.

Flagger expects results.[0].result in the response and NewRelic returns results.[0].any_string as detailed in their documentation and the tests.

As an example, two JSON retrieved from Flaggers log (i removed non-essential data but maintained the same structure) where the result changes depending on the function used, and using aliases on the query does not change the results filed.

Flagger logged JSON with sum query:

{
    "metadata": {
        "accounts": [
            111111
        ],
        "contents": [
            {
                "attribute": "numeric(test_amount)",
                "function": "sum",
                "simple": true
            }
        ]
    },
    "results": [
        {
            "sum": 91979529
        }
    ]
}

Flagger logged JSON with Count query and alias as 'result':

{
    "metadata": {
        "accounts": [
            111111
        ],
        "contents": [
            {
                "alias": "result",
                "contents": {
                    "function": "count",
                    "simple": true
                },
                "function": "alias"
            }
        ]
    },  
    "results": [
        {
            "count": 923732
        }
    ]
}

Flagger's metric template is a simple count query

apiVersion: flagger.app/v1beta1
kind: MetricTemplate
metadata:
  name: newrelic-log-test
  namespace: flagger
spec:
  provider:
    type: newrelic
    secretRef:
      name: newrelic
  query: |
    SELECT count(*) as 'result' FROM Log

Log from the sum test

16:58:59.927 - Halt advancement no values found for custom metric: newrelic-log-test: invalid response: {"results":[{"sum":9.1979529E7}],"performanceStats":{"inspectedCount":1244054,"omittedCount":0,"matchCount":431023,"wallClockTime":101,"exceedsRetentionWindow":false},"metadata":{"accounts":[111111],"eventTypes":["Log"],"eventType":"Log","openEnded":true,"beginTime":"2024-01-22T19:56:59Z","endTime":"2024-01-22T19:58:59Z","beginTimeMillis":1705953419756,"endTimeMillis":1705953539756,"rawSince":"120 SECONDS AGO","rawUntil":"NOW","rawCompareWith":"","guid":"11111111-1111-1111-1111-111111111111","routerGuid":"11111111-1111-1111-1111-111111111111","messages":[],"contents":[{"function":"sum","attribute":"numeric(response_code)","simple":true}]}}: no values found
16:59:29.189 - Rolling back myapp.api failed checks threshold reached 2
16:59:29.380 - Canary failed! Scaling down myapp.api

To Reproduce

• Configure NewRelic credentials according to the documentation
• Create a MetricTemplate with the code provided
• Add the thresholds on the app canary

apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
  name: myapp
spec:
  analysis:
    metrics:
      - name: newrelic-log-test
        templateRef:
          name: newrelic-log-test
          namespace: flagger
        thresholdRange:
          min: 100
        interval: 1m

• Execute a canary deployment

Expected behavior

Flagger reading the response despite of the json key name.

Additional context

• Flagger version: 1.35.0
• Kubernetes version: 1.26.11
• Service Mesh provider: Istio

[rp-martinbarmaymon]

I created this PR to solve the issue https://github.com/fluxcd/flagger/pull/1585