REQUEST: Need for AWS account for hosting CAPA generated AMIs

[Ankitasw]

We need to host all of the new CAPA AMIs going forward in a CNCF account such that it's maintained upstream. Currently, the cost for running EC2 instances to generate AMIs is average of 5K USD in last 6 months. We also need small amount of budget for running a lambda function(costing around 20USD) and data transfer(costing around 36USD).

Would it be possible to provide a separate ACL to make this happen?

Refer slack thread here for more details.

[Ankitasw]

cc @richardcase @dims

[Ankitasw]

Hey @dims 👋 Can we prioritize this?

[dims]

+1 from me!

[ameukam]

@Ankitasw who owns the current account ? we can transfer it under the community funding if needed.

[Ankitasw]

VMware owns the current account and hence we need a new one such that others also has access to the account

[hh]

I'm not sure if it's the amount we are looking for, but I did notice we migrated "cncf-k8s-infra-aws-capa-ami" account from the CNCF AWS Org to Kubernetes AWS AWS Org a while back: https://github.com/kubernetes/k8s.io/issues/4626#orgbead97b

The password for arn:aws:organizations::348685125169:account/o-kz4vlkihvy/819546954734 is stored in the Kubernetes Community 1Password vault AWS CI Accounts.

export AWS_PROFILE=hh@kubernetes
aws organizations describe-account --account-id 819546954734

{
    "Account": {
        "Id": "819546954734",
        "Arn": "arn:aws:organizations::348685125169:account/o-kz4vlkihvy/819546954734",
        "Email": "[email protected]",
        "Name": "cncf-k8s-infra-aws-capa-ami",
        "Status": "ACTIVE",
        "JoinedMethod": "INVITED",
        "JoinedTimestamp": "2023-02-07T01:11:07.090000+02:00"
    }
}

[hh]

@dims / @ameukam Here is the 1Password Kubernetes community Vault and entry for the account. Share as you see fit. I'd like to find a way to manage ongoing access to 1Password Vault entries so passwords can be updated and shared more easily.

• Check Kubernetes Vaults
• Inspect “AWI CI accounts”
• Find capa-ami password item
• Retrieve capi-ami one password item
• Share this link (good for 7 days)

Check Kubernetes Vaults

op vault list

ID                            NAME
5qq5hxaboazxl5p4e5dr6gypqi    Private
atxptplouln57mbc5kvt3tq6jm    AWS CI accounts
x5q2rsmkygy56vwmrgsccuoi2a    Shared

Inspect “AWI CI accounts”

op vault get "AWS CI accounts"

ID:                   atxptplouln57mbc5kvt3tq6jm
Name:                 AWS CI accounts
Type:                 USER_CREATED
Attribute version:    1
Content version:      125
Items:                87
Created:              4 months ago
Updated:              1 month ago

Find capa-ami password item

op items list | grep capa-ami

vatlttczb3iebcmf7t5hlartwq    Awazon (cncf-k8s-infra-aws-capa-ami)                           AWS CI accounts            3 months ago

Retrieve capi-ami one password item

op items get vatlttczb3iebcmf7t5hlartwq | grep -v password:

ID:          vatlttczb3iebcmf7t5hlartwq
Title:       Awazon (cncf-k8s-infra-aws-capa-ami)
Vault:       AWS CI accounts (atxptplouln57mbc5kvt3tq6jm)
Created:     3 months ago
Updated:     3 months ago by Riaan
Favorite:    false
Tags:        aws,production,sig-k8s-infra
Version:     1
Category:    LOGIN
Fields:
  username:    [email protected]
URLs:
  website:    https://signin.aws.amazon.com (primary)

Share this link (good for 7 days)

We could do this, but it’s only good for 7 days and then access to this is gone. I’d like to find a process to share this longer term.

op items get vatlttczb3iebcmf7t5hlartwq --share-link

[dims]

@hh thank you I can confirm that i see it in my 1password

[dims]

@Ankitasw please DM me, we'll work out what you need (looks like you want to install and run a lambda at least!)

[ameukam]

@hh Can we instead create an dedicated user for @Ankitasw with AdministratorAccess on the account cncf-k8s-infra-aws-capa-ami ? Thanks!

[richardcase]

@ameukam & @hh - Would it also be possible to add me as well? So we have 2 maintainers of CAPA on the account.

[ameukam]

@ameukam & @hh - Would it also be possible to add me as well? So we have 2 maintainers of CAPA on the account.

+1 from me.

[hh]

• Our top level [email protected] account + org
• List of current top level IAM Users
• Inspect [email protected] account

The AWS account [email protected] / arn:aws:organizations::348685125169:account/o-kz4vlkihvy/819546954734 is probably where we want to create the iam roles. Probably similar to the way we create them at the top level Kubernetes AWS account.

• See https://github.com/kubernetes/k8s.io/issues/5010#issuecomment-1580212057

However I’m not sure how we want to manage the terraform for AWS org member-accounts, and the resources (like IAM users) that are needed by them.

Should the CAPA team create their own terraform to create acccounts there? I can’t imagine scaling this if we have to manage all-the-k8s sub/member-acccounts with terraform in the same repo managed by the same team.

Suggestions welcome.

If we decide to go with top level accounts, and a good read might be Accessing member accounts in your organization, however I would still recommend we find a way to delegate or help setup a separate way to manage the AWS K8s organization member-account terraform.

Our top level [email protected] account + org

aws organizations describe-organization

{
    "Organization": {
        "Id": "o-kz4vlkihvy",
        "Arn": "arn:aws:organizations::348685125169:organization/o-kz4vlkihvy",
        "FeatureSet": "ALL",
        "MasterAccountArn": "arn:aws:organizations::348685125169:account/o-kz4vlkihvy/348685125169",
        "MasterAccountId": "348685125169",
        "MasterAccountEmail": "[email protected]",
        "AvailablePolicyTypes": [
            {
                "Type": "SERVICE_CONTROL_POLICY",
                "Status": "ENABLED"
            }
        ]
    }
}

List of current top level IAM Users

aws iam list-users --output=table --query 'Users[*].[UserName,Arn]'

---------------------------------------------------------------
|                          ListUsers                          |
+-------------+-----------------------------------------------+
|  arnaud     |  arn:aws:iam::348685125169:user/arnaud        |
|  bentheelder|  arn:aws:iam::348685125169:user/bentheelder   |
|  dims       |  arn:aws:iam::348685125169:user/dims          |
|  hh         |  arn:aws:iam::348685125169:user/hh            |
|  jeefy      |  arn:aws:iam::348685125169:user/jeefy         |
+-------------+-----------------------------------------------+

Inspect [email protected] account

aws organizations describe-account --account-id 819546954734

{
    "Account": {
        "Id": "819546954734",
        "Arn": "arn:aws:organizations::348685125169:account/o-kz4vlkihvy/819546954734",
        "Email": "[email protected]",
        "Name": "cncf-k8s-infra-aws-capa-ami",
        "Status": "ACTIVE",
        "JoinedMethod": "INVITED",
        "JoinedTimestamp": "2023-02-07T00:11:07.090000+01:00"
    }
}

[richardcase]

Should the CAPA team create their own terraform to create acccounts there? I can’t imagine scaling this if we have to manage all-the-k8s sub/member-acccounts with terraform in the same repo managed by the same team.

@hh (and @Ankitasw ) It would be good if there was a way for the maintainers of CAPA to manage the access to the accounts they use (the one for AMIs in this case). It would be more scalable as a general principal.

I'd be happy maintaining terraform, or perhaps yaml for ACK for the CAPA specific account.

[richardcase]

Circling back to this. We need to build & publish some new AMIs to the AWS account for CAPA. How should we manage access to the account for the CAPA maintainers?

@hh @dims - any thoughts on this?

[richardcase]

The current list of maintainers of CAPA who will ideally need to be able to publish new AMIs are:

• @richardcase
• @Ankitasw
• @dlipovetsky
• @vincepri

(source)

This changes over time so happy to contribute terraform or something else if needed.

[BobyMCbobs]

I think that the best path is to create terraform in infra/aws/terraform/cncf-k8s-infra-aws-capa-ami, where the Terraform provider for AWS is either expected to be run in the account or that it is accessing through assume role with the OrganizationAccountAccessRole for full-permissions. In that terraform, specifying the IAM users stated above. https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user https://github.com/kubernetes/k8s.io/pull/5044

cc @hh

[richardcase]

Thanks @BobyMCbobs .

I have started work on the terraform based on your suggestions and will have a PR for it soon.

[richardcase]

Finally got around to updating the PR for this.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[richardcase]

This is still needed:

/remove-lifecycle stale