cluster-api-provider-aws icon indicating copy to clipboard operation
cluster-api-provider-aws copied to clipboard

✨ feat: Add IRSA support for self-managed clusters (rebase)

Open sl1pm4t opened this issue 1 year ago • 29 comments
trafficstars

What type of PR is this?

/kind feature

What this PR does / why we need it:

Adds IRSA functionality to self-managed clusters. This will bring the self-managed clusters inline in functionality with Managed clusters that provide most of this functionality out of the box.

With this PR, the following new resources are created:

  • S3 bucket ( if not already created for the Ignition feature )
  • Two public world readable objects in the S3 bucket:
    • <cluster_name>/.well-known/openid-configuration - OpenID Connect discovery document
    • <cluster_name>/openid/v1/jwks - Service Account signing public key
  • AWS IAM Identity Provider, configured to trust the Issuer found at the S3 URL where the OIDC discovery doc and keys are published.

This is a continuation of an old unmerged PR #4094 - with some fixes and some functionality removed to reduce the scope of the PR.

The functionality removed includes:

  • This PR no longer deploys the amazon-pod-identity-webhook addon to the workload cluster. I felt there are already many ways to manage cluster addons, including ClusterResourceSets or CAAPH, and that it was unnecessary to install the addon via the controller which then becomes an ongoing maintenance burden. Instead, the requirement for the addon has been added to the documentation.
  • This PR no longer modifies the API Server service-account-issuer argument through kubeadm patches. This is easily covered in the documentation and only requires a single line of config to be added to the AWSCluster resource, but also during testing I experienced issues with this being applied inconsistently, resulting in different values across the control plane nodes.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):

Fixes #3560 Supersedes #4094

Special notes for your reviewer:

This PR adds a new ReconcileOIDCProvider to the AWSCluster reconciliation loop.

  • Created a new IAM service, in the future the same logic for EKS could be combined as previously the OIDC code was buried in the EKS service. Details on the reconciler can be found in comments.
  • Extend the S3 service to allow uploading data to arbitrary keys, that can also be marked as public (OIDC discovery docs need to be public for 3rd party systems to retrieve public keys etc.)
  • Exposed a ManagementClient and RemoteClient for both cluster types and exported Client.
  • Moved OIDCProvider status type to v1beta2 and migrated out of the EKS API to make one type both clusters can reference a single type.
  • This PR adds a new Experimental feature flag to enable this functionality. This feature is dependent on the S3 bucket associated with Ignition node configuration, but it felt unintuitive to need to enable the Ignition feature flag to get OIDC support.

Checklist:

  • [X] squashed commits
  • [X] includes documentation
  • [x] includes emojis
  • [ ] adds unit tests
  • [ ] adds or updates e2e tests

Release note:

Add IRSA support for self-hosted clusters

sl1pm4t avatar Aug 28 '24 02:08 sl1pm4t