cluster-api-provider-aws icon indicating copy to clipboard operation
cluster-api-provider-aws copied to clipboard

✨ Support BYO Public IPv4 Pool when provision infrastructure

Open mtulio opened this issue 1 year ago • 12 comments
trafficstars

What type of PR is this?

/kind feature /kind api-change /kind documentation

What this PR does / why we need it:

Introducing support of PublicIpv4Pool to provision base cluster infrastructure consuming public IPv4 (EIP) from a custom Public IPv4 pool brought to AWS.

A subset of changes of this PR is isolated into those PRs:

  • [x] https://github.com/kubernetes-sigs/cluster-api-provider-aws/pull/4899
  • [x] https://github.com/kubernetes-sigs/cluster-api-provider-aws/pull/4900

Which issue(s) this PR fixes

Fixes #4887

Special notes for your reviewer:

Checklist:

  • [ ] squashed commits
  • [ ] includes documentation (WIP)
  • [ ] includes emojis
  • [ ] adds unit tests
  • [ ] adds or updates e2e tests (NA)

Release note:

Introduce the support of provisioning public IPv4 address consuming from a custom Public IPv4 Pool that is brought to AWS. When the field `PublicIpv4Pool` is set with the pool ID, all the network resources which claims public IPv4, such as Network Load Balancers and NAT Gateways, will be created consuming from the custom pool.

mtulio avatar Apr 03 '24 19:04 mtulio

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please assign justinsb for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar Apr 03 '24 19:04 k8s-ci-robot

Hi @mtulio. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Apr 03 '24 19:04 k8s-ci-robot

prevent WIP notifications /uncc Ankitasw dlipovetsky

mtulio avatar Apr 29 '24 22:04 mtulio

/test pull-cluster-api-provider-aws-e2e

mtulio avatar Apr 30 '24 05:04 mtulio

@mtulio: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test pull-cluster-api-provider-aws-e2e

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Apr 30 '24 05:04 k8s-ci-robot

HI @damdo @nrb - can I have labels to test in this PR? Thanks

mtulio avatar Apr 30 '24 05:04 mtulio

/ok-to-test

damdo avatar Apr 30 '24 07:04 damdo

@mtulio: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-cluster-api-provider-aws-test 4c41955afcb2e2bce07af1754e4c34226c07debe link true /test pull-cluster-api-provider-aws-test
pull-cluster-api-provider-aws-verify 4c41955afcb2e2bce07af1754e4c34226c07debe link true /test pull-cluster-api-provider-aws-verify

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

k8s-ci-robot avatar Apr 30 '24 07:04 k8s-ci-robot

/test pull-cluster-api-provider-aws-e2e

mtulio avatar Apr 30 '24 10:04 mtulio

Downstream OpenShift e2e is passing. The downstream is setting PublicIpv4Pool both for infrastructure (VPCSpec) and Machine consuming from an Public IPv4 Pool brought to CI account. Starting tests for regular e2e:

/test pull-cluster-api-provider-aws-e2e /test pull-cluster-api-provider-aws-e2e-eks

mtulio avatar May 09 '24 03:05 mtulio

e2e is mostly passing (eks seems to be flake), added docs and fixed the verify:

/test pull-cluster-api-provider-aws-e2e /test pull-cluster-api-provider-aws-e2e-eks

mtulio avatar May 09 '24 05:05 mtulio

This PR is ready for review. We have some spots to improve the unit tests, but I would like to know if this approach is feasile by maitainers to cover the request #4887 .

The e2e had timeouts, e2e-eks pass, trying again with fix in verify.

/test pull-cluster-api-provider-aws-e2e /test pull-cluster-api-provider-aws-e2e-eks

Please take a look? /cc @damdo @nrb @r4f4

mtulio avatar May 10 '24 11:05 mtulio

/test pull-cluster-api-provider-aws-e2e /test pull-cluster-api-provider-aws-e2e-eks

mtulio avatar May 11 '24 05:05 mtulio

The commits/changes I am performing is improving/adding unit tests, there is not block for review.

@damdo @nrb Would you mind adding it to the review queue, please?

mtulio avatar May 13 '24 15:05 mtulio

/assign @damdo @nrb

mtulio avatar May 13 '24 17:05 mtulio

FWIW I just ran a chaos scenario by fulfilling the custom CIDR pool and run the installation with the fallback to Amazon-pool. I found one better approach to check the fallback, just fixed in the last commit. The details of the tests and results are available in this comment: https://github.com/openshift/installer/pull/8175#issuecomment-2111229833

mtulio avatar May 14 '24 22:05 mtulio

/test pull-cluster-api-provider-aws-e2e /test pull-cluster-api-provider-aws-e2e-eks

mtulio avatar May 15 '24 03:05 mtulio

/test pull-cluster-api-provider-aws-e2e

damdo avatar May 15 '24 06:05 damdo

It looks like the e2e failed on teardown:

/test pull-cluster-api-provider-aws-e2e

damdo avatar May 15 '24 08:05 damdo

Hey @mtulio thanks a lot for this PR! I've left some comments, mostly nitpicks, catching typos, and some questions, nothing really wrong with it! :)

Hello @damdo , thanks a lot for your review, good suggestions. All applied. I am leaving only two comments "unresolved" to further discussion. Let me know your thoughts.

mtulio avatar May 15 '24 13:05 mtulio

/lgtm

vr4manta avatar May 15 '24 13:05 vr4manta

@vr4manta: changing LGTM is restricted to collaborators

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar May 15 '24 13:05 k8s-ci-robot

Hi CAPA maintainers, I will be OoO in the next three weeks, please also ping @rvanderp3 to align/address items in your review in this PR. Thanks.

mtulio avatar May 16 '24 17:05 mtulio

Let's get an approver review. Could any of you PTAL /assign @richardcase @Ankitasw @dlipovetsky @vincepri @nrb

Thanks!

damdo avatar May 23 '24 17:05 damdo

CLA Signed

The committers listed above are authorized under a signed CLA.

  • :white_check_mark: login: mtulio / name: Marco Braga (2f230b38348af971c2bb4f6214d620f8774f1e1a)

/test pull-cluster-api-provider-aws-test

jcpowermac avatar Jun 04 '24 12:06 jcpowermac

/test pull-cluster-api-provider-aws-e2e-eks /test pull-cluster-api-provider-aws-e2e /test pull-cluster-api-provider-aws-e2e-clusterclass /test pull-cluster-api-provider-aws-e2e-conformance

damdo avatar Jun 07 '24 20:06 damdo

Thanks @jcpowermac and @rvanderp3 for taking care of this PR addressing the review while I was out.

/test pull-cluster-api-provider-aws-e2e-eks /test pull-cluster-api-provider-aws-e2e /test pull-cluster-api-provider-aws-e2e-clusterclass /test pull-cluster-api-provider-aws-e2e-conformance

Hey @damdo - apologies I killed the jobs while addressing this comment. Let me trigger again:

/test pull-cluster-api-provider-aws-e2e-eks /test pull-cluster-api-provider-aws-e2e /test pull-cluster-api-provider-aws-e2e-clusterclass /test pull-cluster-api-provider-aws-e2e-conformance

mtulio avatar Jun 07 '24 20:06 mtulio

/test pull-cluster-api-provider-aws-e2e-eks

mtulio avatar Jun 07 '24 22:06 mtulio

/test pull-cluster-api-provider-aws-e2e-eks

Unrelated issue:

STEP: Event details for AWSIAMUserBootstrapper : Resource: AWS::IAM::User, Status: CREATE_FAILED, Reason: Resource handler returned message: "Resource of type 'AWS::IAM::User' with identifier 'bootstrapper.cluster-api-provider-aws.sigs.k8s.io' already exists." (RequestToken: fb13356d-5ac6-1c5f-1b83-6ea7aad87fb4, HandlerErrorCode: AlreadyExists) @ 06/07/24 23:02:31.865

STEP: Event details for cluster-api-provider-aws-sigs-k8s-io : Resource: AWS::CloudFormation::Stack, Status: ROLLBACK_IN_PROGRESS, Reason: The following resource(s) failed to create: [AWSIAMManagedPolicyControllers, AWSIAMInstanceProfileControllers, AWSIAMInstanceProfileNodes, AWSIAMInstanceProfileControlPlane, AWSIAMManagedPolicyControllersEKS, AWSIAMUserBootstrapper]. Rollback requested by user. @ 06/07/24 23:02:31.865

Looking the IAM resource name "Resource of type 'AWS::IAM::User' with identifier 'bootstrapper.cluster-api-provider-aws.sigs.k8s.io' already exists it seems not to be specific for this job run, it makes me wonder if that job supports parallel executions. cc @damdo @nrb

I will trigger again with a, hopefully, low traffic period (Friday night =] ).

/test pull-cluster-api-provider-aws-e2e-eks

mtulio avatar Jun 08 '24 04:06 mtulio

Thanks for your diligence and patience with this @mtulio!

/approve

nrb avatar Jun 12 '24 02:06 nrb

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: nrb

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar Jun 12 '24 02:06 k8s-ci-robot