Support publishing CI artifacts to S3

[rifelpet]

Ref: https://github.com/kubernetes/kops/issues/16637 https://github.com/kubernetes/k8s.io/issues/5127 https://github.com/kubernetes/k8s.io/issues/2625

More context in https://github.com/kubernetes/k8s.io/pull/6950#issue-2393322023

[k8s-ci-robot]

Skipping CI for Draft Pull Request. If you want CI signal for your change, please convert it to an actual PR. You can still manually trigger a test run with /test all

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please ask for approval from rifelpet. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

@rifelpet: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-kops-e2e-k8s-aws-calico-k8s-infra	35c1d03211d182c3005575e2751ee60da382c7c8	link	true	/test pull-kops-e2e-k8s-aws-calico-k8s-infra

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[upodroid]

We want kubetest2-kops to dynamically create a temporary s3 bucket when the presubmit is running, similar to this code https://github.com/kubernetes/kops/blob/master/tests/e2e/kubetest2-kops/gce/gcs.go

I made the IAM changes, but it allows presubmits to write to our CI bucket which isn't ideal(existing behaviour we should change).

[rifelpet]

Yes I see the migrated job on your PR passed, using k8s-staging-kops:

https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kops/16648/pull-kops-e2e-k8s-aws-calico-k8s-infra/1809653969442574336

I believe this means we could migrate all presubmit jobs to use this, but you're right giving presubmit jobs write permissions to the same bucket used for post-merge artifacts like version markers is not great.

We have the k8s-infra-kops-ci-results bucket created in https://github.com/kubernetes/k8s.io/pull/2678, is that in the correct project and can we use it for either presubmit artifacts or version markers, and then revoke permissions for presubmits to whichever bucket we choose for version markers?

[upodroid]

For now, let's retain the existing behaviour as AWS presubmits have been writing to the k8s-staging-kops bucket and let's try to switch to dynamic buckets on AWS.

As for gs://k8s-infra-kops-ci-results, I don't want to use this bucket and we should delete it.

[upodroid]

We do need to ship this to fix the AWS presubmits publishing to the k8s-staging-kops bucket. We carried over that practice when the jobs were migrated to the community cluster.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Mark this PR as fresh with /remove-lifecycle stale
• Close this PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Mark this PR as fresh with /remove-lifecycle rotten
• Close this PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Reopen this PR with /reopen
• Mark this PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Reopen this PR with /reopen
• Mark this PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.