feat(event-hubs-scaler): Add pod/workload identity support to checkpoint in Azure Blob Storage

[andyatwork]

Add support for pod identity based authentication to blob storage accounts in Event Hub scaler

Checklist

• [X] Commits are signed with Developer Certificate of Origin (DCO - learn more)
• [X] Tests have been added
• [ ] A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
• [X] A PR is opened to update the documentation on (repo) (if applicable)
• [X] Changelog has been updated and is aligned with our changelog requirements

Relates to #3569

Relates to kedacore/keda-docs#905

[v-shenoy]

I checked this PR out and tested it. Confirmed that it works with Workload Identity as well @tomkerkhove @andyatwork

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 0
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: eventhub-ta
spec:
  podIdentity:
    provider: azure-workload
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: eventhub-so
spec:
  scaleTargetRef:
    name: nginx-deployment
  minReplicaCount: 0
  maxReplicaCount: 5
  triggers:
  - type: azure-eventhub
    metadata:
      storageAccountName: testhubstorage
      checkpointStrategy: azureFunction
      eventHubNamespace: test-hub-ns
      eventHubName: hub-1
    authenticationRef:
      name: eventhub-ta

[andyatwork]

I checked this PR out and tested it. Confirmed that it works with Workload Identity as well @tomkerkhove @andyatwork

apiVersion: apps/v1
kind: Deployment
metadata:

...

Excellent, thanks for testing this!

[andyatwork]

@tomkerkhove , can you follow up on the change request above?

[tomkerkhove]

Not from my side

[andyatwork]

Not from my side

Sorry, I had responded to your comment inline above and realized I hadn't published my comments yet. They remained "pending", so I guess you hadn't actually seen my response. Hope the replies addresses your concerns.

[v-shenoy]

@JorTurFer Any changes here after upgrading the event hubs SDK?

[JorTurFer]

I didn't touch the eventhub sdk, only the service-bus sdk which I'd say is different

[JorTurFer]

I have checked the SDK for this and, oh surprise, uses other authentication mechanism different from the service-bus SDK... Maybe in the future they unify the SDK in a single SDK for azure or at least reuse the authentication part, but atm I think it's a mess

[v-shenoy]

I didn't touch the eventhub sdk, only the service-bus sdk which I'd say is different

I thought you upgraded it to v3, which is why I was wondering if anything broke over there that would require changes here.

[JorTurFer]

I didn't touch the eventhub sdk, only the service-bus sdk which I'd say is different

I thought you upgraded it to v3, which is why I was wondering if anything broke over there that would require changes here.

Lol, let me check because I didn't notice that

[JorTurFer]

did you mean with this PR? I only changed the service-bus sdk others are minor versions

[v-shenoy]

did you mean with this PR? I only changed the service-bus sdk others are minor versions

Oh, I just saw the title for this issue, https://github.com/kedacore/keda/issues/2986 and I assumed it was upgrading TO v3. If it's minor versions, then shouldn't cause any issues.

[JorTurFer]

No no, it's a minor version, but that minor version created conflicts I think because it uses generics somewhere. That's why I created the issue to remember bumping it after update golang version

[JorTurFer]

BTW; I have checked and there is a beta version for event-hub in the azure-sdk-for-go so eventually we could migrate to it once it's stable and simplify the pod identities reusing the approach of service-bus 🤞

[JorTurFer]

@andyatwork , could you rebase your branch? 🙏 We added e2e test for event hub some weeks ago and rebasing the branch we can execute them before merging the PR

[andyatwork]

Yes, will rebase and update the PR.

[JorTurFer]

/run-e2e event_hub* Update: You can check the progress here