[Spike] Investigate Manage Identity AKS interoperability with CSI

[NathanielRose]

As a: Operator

I want: Better Understanding of Kubernetes-Secrets-Store-CSI-Driver integration with Key Vault on AKS

So that: I can better determine the best secrets solution in an MSI AKS implementation for handling service secrets.

Describe the solution you'd like: Documentation around benefits of using CSI and comparison with FlexVolume.

Acceptance Criteria:

• [ ] CSI vs FlexVolume
• [ ] CSI + MSI & AKS
• [ ] CSI guidance with Key Vault and accessing secrets through Env Vars
• [ ] Terraform implementation of CSI

Describe alternatives you've considered:

• Continue with FlexVolume

Additional context: Related to: https://github.com/microsoft/bedrock/issues/1197

Does this require updates to documentation?: Yes

[jsturtevant]

@paulbouwer just did a deep dive on CSI-driver with MI and could provide so insight

[paulbouwer]

Have a look at the following:

https://github.com/paulbouwer/experiments/blob/master/aks/install-aadpodidentity-and-secretsstoredriver.md

This guidance assumes AKS with Managed Identity (Kubernetes Cloud Provider Identity). It leverages AAD Pod Identity to assign another Managed Identity (Key Vault Identity) to the Secrets Store CSI Driver. This all works.

If you are looking to spin up Secrets Store CSI Driver with a Managed Identity not controlled by AAD Pod Identity, you will have to wait for Azure/secrets-store-csi-driver-provider-azure PR #46 - Add support for MSI to land.