Blue Green Deployment | Error: "no values found for kubernetes metric request-success-rate"

[hverma95]

Describe the bug

We need to configure the Blue-Green deployment strategy (provider = kubernetes) BUT seems that the Prometheus/flagger can't fetch/scrape the request-success-rate metric OR maybe our application isn't exporting this metric.

But when we use the Canary deployment strategy, it works completely fine i.e. provider = nginx

We are using the flagger's Prometheus

Below is the canary custom resource for blue-green deployment:

apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
  annotations:
    meta.helm.sh/release-name: my-app
    meta.helm.sh/release-namespace: app-namespace
  finalizers:
  - finalizer.flagger.app
  generation: 2
  labels:
    app.kubernetes.io/managed-by: Helm
  name: my-app
  namespace: app-namespace
spec:
  analysis:
    interval: 10s
    metrics:
    - interval: 1m
      name: request-success-rate
      thresholdRange:
        min: 99
    threshold: 5
    webhooks:
    - metadata:
        cmd: curl https://my-app.example.com/health | grep UP
        type: bash
      muteAlert: false
      name: acceptance-test
      timeout: 30s
      type: pre-rollout
      url: http://flagger-loadtester.app-namespace/
    - metadata:
        cmd: hey -z 1m -q 10 -c 2 https://my-app.example.com/
      muteAlert: false
      name: load-test
      timeout: 5s
      type: ""
      url: http://flagger-loadtester.app-namespace/
  progressDeadlineSeconds: 1200
  provider: kubernetes
  revertOnDeletion: true
  service:
    port: 80
    portDiscovery: true
    targetPort: 3545
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app

To Reproduce

Expected behavior

The request-success-rate metric should be exported and collected successfully and metric checks should pass.

We need to use the blue-green strategy only and need to know how the metric named request-success-rate will be generated by our application so that based on this metric's success, the application gets promoted.

Additional context

• Flagger version: flagger-1.29.0
• Kubernetes version: 1.22
• Service Mesh provider:
• Ingress provider: NGINX Ingress Controller

[jiri-pinkava]

Might this be related to #421 ? Would providing custom metric template with modification as described here solve this issue? https://github.com/fluxcd/flagger/issues/421#issuecomment-597870052

[mitangelo]

if we provide a custom metric template do we need to have prometheus operator externally deployed?