test-infra

test-infra copied to clipboard

What should be cleaned up or changed:

We are moving the /prow directory to a new Prow source repo under sigs.k8s.io.

The migration should satisfy the following requirements:

• Developers use a dedicated Prow repository on GitHub.
• The Prow repository must not depend on libraries in the test-infra repo.
• Prow commit history is retained in the new repository.
• Presubmit and postsubmit Prow jobs such as image publishing will run on the new repository.

Provide any links for context:

We are moving the /prow directory to a new Prow source repo under sigs.k8s.io.

Do we have an approved plan yet by the owners?

I think this is more aspirational than definite yet.

We are using prow inside our CI (kubevirt.io) and would want to consume updates from the u/s

We have seen the announcement on LWKD to move prow out of test_infra, it points towards this document: https://docs.google.com/document/d/1f-2qcUD1K3xRV2FXJSf5G0aAMbALq709O9ZwYKKj0QM/edit

What do I need to do to get access to it?

@dhiller That document is incomplete and the author didn't transfer permissions before leaving the company so I can't easily share it.

Lets use this GH issue to share any relevant info instead since the meat of the work to migrate the source to the kubernetes-sigs/prow repo is now complete and we're just waiting to officially switch the source of truth for development.

Here is the draft of the communication I'm planning to send out to the community (once I've had it reviewed). Hopefully this answers any questions you have.

---

Subject: Prow Source Code Migration on April 9th

If you do not develop Prow or GHProxy in the kubernetes/test-infra repo and you do not depend on any of their go packages, you can stop reading now. We are finally ready to move the Prow (and GHProxy) source code out of kubernetes/test-infra and into the kubernetes-sigs/prow repo! The source code has actually already been copied there (with commit history) and all ProwJobs have been created and tested except for the container image publishing postsubmit. On April 9th we will make the official switch and kubernetes-sigs/prow will become the source of truth for Prow and GHProxy development and container images will be published from there.. From this point on I’ll just refer to "Prow", but this all applies to GHProxy as well.

How this impacts you and what to do:

• If you just use Prow or maintain a Prow instance there is nothing you need to do. Container images will still be available at gcr.io/k8s-prow/.
• If you don't develop Prow itself, but you do rely on its go packages (via importing the k8s.io/test-infra go module), after April 9th you'll need to update import statements to be prefixed with 'sigs.k8s.io/prow' rather than 'k8s.io/test-infra'. Since the repo relative paths will remain the same, you can use a sed command like this one to fix any references: sed -i 's,k8s.io/test-infra/prow,sigs.k8s.io/prow/prow,g;s,k8s.io/test-infra/ghproxy,sigs.k8s.io/prow/ghproxy,g' Don't forget to run go mod tidy afterwards.
• If you do develop Prow you'll need to start targeting PRs against the kubernetes-sigs/prow repo instead of kubernetes/test-infra. You can transfer any open PRs to the new repo by using the git format-patch and git am commands. You'll also need to update package references as described in the previous bullet point.

Plans for the switch over date (April 9th):

The actual plan for the switch over is as follows:

1. Delete the Prow container image publishing postsubmit from kubernetes/test-infra and uncomment the config for the corresponding job in kubernetes-sigs/prow.
2. Make a noop change to kubernetes-sigs/prow to trigger the container image build and ensure that it succeeds. If there is a problem we revert the previous step, if there is no problem we have successfully made the switch and the source of truth for Prow development is now kubernetes-sigs/prow.

The following steps (also for April 9th) will clean up kubernetes/test-infra:

4. Delete the prow/ and ghproxy/ directories in kubernetes/test-infra to prevent further development.
5. Delete any other tooling that was only needed for Prow. At this time, the only thing I know of is 'hack/ts-rollup'.
6. Simplify some 'run_if_changed' expressions that no longer need to reference Prow.

If you have any questions or concerns please speak up on this GitHub issue: https://github.com/kubernetes/test-infra/issues/31728

If you just use Prow or maintain a Prow instance there is nothing you need to do. Container images will still be available at gcr.io/k8s-prow/.

This is actually a problem that we need to tackle seperately.

A) GCR is deprecated B) Serving from a single region registry like this is not cost efficient. Originally it was only internal consumption by the project's own CI deployment.

If we want to publish images for external users it should really be at https://registry.k8s.io (note: images must go through image promotion, you cannot direct push), for internal usage it could be moved to an AR.

Hi, is it possible to store the prow code at the root of the repo instead of inside the prow folder of k-sigs/prow repo?

Hi, is it possible to store the prow code at the root of the repo instead of inside the prow folder of k-sigs/prow repo?

That is possible and something we could choose to change later, but changing the repo relative paths of the source code would cause a lot of breaking changes to the image build so I didn't want to try to manage that while moving the code to another repo.

FWIW I don't think the prow source should be at the root of the repo anyways. It seems cleaner to me to keep the source code in a separate directory from site and hack specifically. I think the main issue is the prow directory name matching the repo name causing import paths like sigs.k8s.io/prow/prow/config. Something like sigs.k8s.io/prow/src/config or sigs.k8s.io/prow/pkg/config seems preferable.

I want to avoid prow/prow in the import path. Happy to make this change after the repo migration

We should make this change before we start versioning prow.

Just to underline:

A) GCR is deprecated

... And will be shut down in less than a year. Prow is unusual in that it doesn't have promoted images currently. Other projects will at most have disruption to their staging => promotion flow if nothing is done. Prow AFAICT will have a mass outage.

Technically unrelated, but if we're setting up image publishing on a new repo, I'd ask that we get ahead of this now.

Technically unrelated, but if we're setting up image publishing on a new repo, I'd ask that we get ahead of this now.

"setting up image publishing on a new repo" just entails running the postsubmit job against the new repo, all the make targets and build tools are in the same repo relative paths and I've already copied over and updated the postsubmit config here: https://github.com/kubernetes/test-infra/blob/5d4571a35499c79f4988baa461f6d136485ea8ab/config/jobs/kubernetes-sigs/prow/prow-postsubmits.yaml#L38-L82

Switching to image promotion will take more work and is not something I can commit to personally now. Since this is orthogonal to the source move, perhaps someone else can take ownership of image promotion?

Hi and thanks for all the work. What about the PRs that have been opened and possibly closed prior April 9th? I mean, opened more or less recently.

Hi and thanks for all the work. What about the PRs that have been opened and possibly closed prior April 9th? I mean, opened more or less recently.

• Any PRs that are merged before April 9th will be included when I sync the commits to kubernetes-sigs/prow on April 9th.
• Any PRs that have not been merged and are still open on April 9th will need to be closed, and reopened against the new repo. You'll also need to update the go module names in your PR. The second and third bullet points under the "How this impacts you" section describe what to do if you are the author of an open PR: https://groups.google.com/g/kubernetes-sig-testing/c/eynyMwLvibw
• Any new PRs after April 9th should be created against the kubernetes-sigs/prow repo. (The prow/ directory will be deleted from kubernetes/test-infra.)

Working on this now. See slack or here for updates: https://kubernetes.slack.com/archives/CDECRSC5U/p1712689517164499

After a bit more tweaking (https://github.com/kubernetes/test-infra/pull/32420) everything seems to be working in both this repo and kubernetes-sigs/prow. The source migration is officially done. Please feel free to resume Prow contributions in kubernetes-sigs/prow now.

If you use gcr.io/k8s-prow/generic-autobumper you may need to add the following to your configuration file to suppress errors about inconsistent image versions: https://github.com/kubernetes/test-infra/blob/0d65b065989a2f85af0d1d2fe2d5c91c60edde9c/config/prow/autobump-config/prow-component-autobump-config.yaml#L25-L35

I want to avoid prow/prow in the import path. Happy to make this change after the repo migration

I opened a PR to fix this. Can we get it merged as soon as possible?

https://github.com/kubernetes-sigs/prow/pull/101

The import path should be fixed now. I sent this message below to the testing mailing list that is pending approval.

Hello Everyone

We merged a change today in the prow repository that eliminates prow being repeated twice in the import path.

The code that lived in sigs.k8s.io/prow/prow and sigs.k8s.io/prow/ghproxy have now been moved to the following locations:

• sigs.k8s.io/prow/prow/cmd/* packages have been moved to sigs.k8s.io/prow/cmd/*
• sigs.k8s.io/prow/prow/pkg/* packages have been moved to sigs.k8s.io/prow/pkg/*
  • Exceptions:
    • sigs.k8s.io/prow/prow/pkg/external-plugins/* packages have been moved to sigs.k8s.io/prow/cmd/external-plugins/* . Have a look at this commit for details
• sigs.k8s.io/prow/ghproxy has moved to sigs.k8s.io/prow/cmd/ghproxy and the libraries have been moved to sigs.k8s.io/prow/pkg. Have a look at this commit for full details

https://pkg.go.dev/sigs.k8s.io/[email protected] This is the commit and version of the package where this change has been applied to.

Thank you

Mahamed

Thanks all! This is awesome!

I filed https://github.com/kubernetes-sigs/prow/issues/113 to track the image hosting tangent.