Add new package `auth` for various authentication scenarios

[aryan9600]

Add a new package auth which handles multiple authentication use cases for various cloud providers and SaaS. It inculdes the following sub packages:

• aws: For AWS and ECR auth
• gcp: For GCP and GAR auth
• azure: For Azure and ACR auth
• github: For GitHub Apps auth
• registry: For OCI registry auth
• git: For auth against Git providers

The packages auth/registry and auth/git also support caching the authentication credentials via AuthOptions.CacheOptions.Key. Caching needs to explicitly enabled by either calling auth.InitCache() or specifying a cache via AuthOptions.CacheOptions.Cache.

Fixes #642

[darkowlzz]

I tried reviewing this but it's hard to review properly with any certainty without any test actually running against the providers. We have most of the test setup needed for testing registry auth in the OCI integration test package. The whole test setup can be copied to this package or temporarily, the OCI integration test's test app can be modified to also use new auth via a flag and verify that it continues to work against all the providers.

For git, flux2 repo has azure devops and google source repository test infrastructure code. The OCI integration test and flux2 integration test setup are very similar. I think both of them can be combined.

I see that this PR resolves #642 . I think there's a lot of things to be done before that can be resolved. Maybe the tests can be in a separate branch against this PR branch. I think running tests against the providers will be very helpful in reviewing these changes properly. Without that, I don't know how to review this without any certainty about the implementation.

[Poltergeisen]

Is this still planned for the next minor release of flux? I'm looking forward to using this soon if possible to connect to Azure Devops Git.

[b-rand]

Is this still planned for the next minor release of flux? I'm looking forward to using this soon if possible to connect to Azure Devops Git.

Also awaiting this functionality, any updates?

[ageisen2000]

Will this work with the flux bootstrap command when implemented?

[stefanprodan]

Also awaiting this functionality, any updates?

We used to have an Azure account to test all of this but MSFT has canceled our subscription. Also https://github.com/fluxcd/flux2/discussions/4544

[evandam]

I'm just wondering if it would be easier/more manageable if this was broken down a bit? For example it would probably be easier to test GitHub App auth and making related changes to the GitHub provider without worrying about AWS/Azure/GCP mixed in too?

Granted I'm a bit biased since I opened https://github.com/fluxcd/flux2/discussions/4356 and would love to see it implemented. No worries though and appreciate the effort everyone puts in to make Flux awesome!

[AdrianBalcan]

Hi guys, this feature can be a solution differentiator for FluxCD over other solutions. It could bring full automation IaC between terraform AKS deployment and FluxCD which would populate the Kubernetes cluster and removing manual steps.

[yashwanth-l]

Hello!

Any updates further on this issue?

This feature could be really help in avoiding using PAT and SSH Keys (GH doesn't even allow the same keys in multiple repos)

[iamandymcinnes]

Would also love to see this, it would be so nice not to have manage tokens like a neanderthal :)