k8s.io icon indicating copy to clipboard operation
k8s.io copied to clipboard

Published 20 hours ago • verified publisher icon kubernetes

Start Publishing Staging Images to Artifact Registry

Open upodroid opened this issue 2 years ago • 20 comments

Part of: #1343

Notes: https://gist.github.com/upodroid/a33723a7e1abc5e9c6fabc6b07e7aac0

When images are built after a PR is merged, they need to be pushed to Artifact Registry(AR).

~Unlike Google Container Registry(GCR), you don't need a separate GCP project per staging project. The permission boundary is at the repository level instead of the project. This allows multiple registries to be created per project. Therefore, we will create a new project, start publishing images to it and delete the other projects after the transition.~

~We will need to do the following: - create a new GCP project (k8s-artifacts-staging) - create a docker image repository per project - modify the jobs to push images to prow. - Provision the infra with terraform. There is a separate issue open to deploy changes to terraform via Prow on merges.~

Prod changes need to be done via the shell scripts which makes deploying staging via terraform kind of pointless.

Open questions:

  • Do we need to backfill the staging repository via gcrane?

@puerco

/area artifacts /priority important-soon /area release-eng

upodroid avatar Jul 12 '22 16:07 upodroid

After #3968 is merged, we need to do a few things:

  • repos that use ko to build images(k-sigs/bom,etc), modify KO_DOCKER_REPO variable to us-docker.pkg.dev/k8s-staging-bom/images/bom as an example
  • Modify image-builder to tweak the cloudbuild.yaml file

Bit unsure about other things.

upodroid avatar Jul 13 '22 11:07 upodroid

Spoke about this at the sig-testing meeting. Will push it out to the future when GCR deprecation is actually announced by Google.

May want to standardize on Aritfact Registry eventually, but would require tracking down a number of staging registries and changing team push processes (with unclear payoff right now)

/priority backlog /sig testing

upodroid avatar Jul 26 '22 17:07 upodroid

/milestone v1.26

ameukam avatar Aug 25 '22 19:08 ameukam

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Nov 23 '22 20:11 k8s-triage-robot

/remove-lifecycle stale /milestone v1.27 /priority important-longterm

ameukam avatar Dec 21 '22 10:12 ameukam

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Mar 21 '23 11:03 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle rotten
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot avatar Apr 20 '23 11:04 k8s-triage-robot

/remove-lifecycle rotten /milestone v1.29

ameukam avatar Jun 24 '23 10:06 ameukam

Strawman:

  1. Create AR versions of all staging registries. These can be empty, but should have the same IAM access / names as the GCRs
  2. Ask subprojects to switch to the AR registry. They do so by:
    1. Switching new builds to push to AR (this will vary by how they're doing image push)
    2. Running a backfill task we provide (gcrane cp -R)
    3. Switching from GCR to AR in image promoter config

We should probably check in with release engineering about that last part.

We could alternately:

  1. Create AR versions of all staging registries.
  2. SIG K8s Infra spins up a job or jobs to continuously backfill from GCR to AR (grane cp -R ... potentially excessive API usage ...)
  3. SIG K8s Infra swaps over all the promoter manifests to promote from AR
  4. We ask subprojects to stop writing to GCR so we can wind down the grane sync for their repo.

The first plan is less resource intensive but more people-coordination heavy

BenTheElder avatar Jan 18 '24 00:01 BenTheElder

Another alternative could be to transition existing GCR repos to AR repos: https://cloud.google.com/artifact-registry/docs/transition/setup-gcr-repo

The infrastructure changes would be minimal and there is little to be done by the community. The downsize of this approach is mostly cost. we will use multi-regional AR repositories.

ameukam avatar Jan 18 '24 08:01 ameukam

I forgot about this! Nice! We should definitely estimate the cost delta of that approach ...

One other downside: skew versus newly created registries going forward.

BenTheElder avatar Jan 18 '24 18:01 BenTheElder

One other downside: skew versus newly created registries going forward.

AR allow us to create AR repos using gcr.io (traffic is redirected from gcr.io to the AR repo) so I think we are fine until GCR is gone.

ameukam avatar Jan 18 '24 23:01 ameukam

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Apr 18 '24 00:04 k8s-triage-robot

Maybe instead tracking in https://github.com/kubernetes/k8s.io/issues/1343

BenTheElder avatar Apr 18 '24 02:04 BenTheElder

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle rotten
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot avatar May 18 '24 02:05 k8s-triage-robot

Not rotten

upodroid avatar Jun 17 '24 20:06 upodroid

Need to go back and create etcd-manager staging bucket in https://github.com/kubernetes/k8s.io/pull/6897

slack conversation : https://kubernetes.slack.com/archives/CCK68P2Q2/p1718654806739639

dims avatar Jun 17 '24 20:06 dims