k8s.io
k8s.io copied to clipboard
kubernetes
Migrate kettle to k8s-infra
Part of migrating away from gcp-project k8s-gubernator: https://github.com/kubernetes/k8s.io/issues/1308
- origin gcp-project: k8s-gubernator
- origin cluster-name: g8r
- apps:
- [ ] kettle
- repo: https://github.com/kubernetes/test-infra/tree/master/kettle
My suggestions for target:
- project: kubernetes-public
- cluster: aaa
- namespace: kettle
/wg k8s-infra /area cluster-infra /sig testing
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale
Migrating kettle most likely looks something like
- migrate the bigquery database kettle writes to (https://github.com/kubernetes/k8s.io/issues/1307)
- make sure the permissions kettle needs against google.com resources work for non-google.com accounts (e.g. pubsub permissions in kubernetes-jenkins)
- migrate kettle
FYI @MushuEE given that you've been modifying kettle lately, if you happen see things that could help inform a plan for this, drop 'em here
When you say
migrate the bigquery database kettle writes to
is that to a new project? What is the: target project and target cluster?
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
- After 90d of inactivity,
lifecycle/staleis applied - After 30d of inactivity since
lifecycle/stalewas applied,lifecycle/rottenis applied - After 30d of inactivity since
lifecycle/rottenwas applied, the issue is closed
You can:
- Mark this issue or PR as fresh with
/remove-lifecycle stale - Mark this issue or PR as rotten with
/lifecycle rotten - Close this issue or PR with
/close - Offer to help out with Issue Triage
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
I don't have much access to the k8s-gubernator project^1 currently, so it's a bit difficult to do much to help here. I'm going to ask around about access.
The deployment details are more or less in the repo at least https://github.com/kubernetes/test-infra/blob/master/kettle/.
@ixdy still works at Google and still had access, despite long since not working in cloud anymore ... myself and @liggitt now have owner access to k8s-gubernator project for continuity until we can migrate it. Thanks Jeff!
This still needs to happen before the prow default cluster shutdown in August and sooner is better.
So we still have one "g8r" cluster on 1.26.11-gke.1055000 with 3 node pools, "pool-1" (e2-highmem-16, 1 node), "pool-highmem" (n1-highmem-8, 2 nodes), "pool-large" (n1-standard-8, 0 nodes).
It is running "kettle" and "kettle-staging" deployments with one pod each.
Each of those has a PD-SSD, 3001 and 201 GB respectively.
There are some bigquery datasets in this project, build/all is 1.67 TB.
Given initially ingest this data from the prow GCS logs, I think we should probably look at cold-starting a new instance running in AAA, just overriding the cluster/project and deploying with the existing tooling.
There's a lot to be desired around auto deployment etc however
I think @dims has this working, one remaining item will be when we're confident this is done let Googlers know and we'll see about turning down the old instance / GCP project ... (FYI @michelle192837 @cjwagner)
[I scaled the old cluster down to zero this week, we'll check back next week]
https://storage.googleapis.com/k8s-triage/index.html is being updated.
and the flakes json looks good as well
❯ gsutil ls -l gs://k8s-metrics
Updates are available for some Google Cloud CLI components. To install them,
please run:
$ gcloud components update
114 2024-05-02T00:05:31Z gs://k8s-metrics/build-stats-latest.json
10040 2024-05-02T00:04:51Z gs://k8s-metrics/failures-latest.json
103224 2024-05-02T00:04:20Z gs://k8s-metrics/flakes-daily-latest.json
204024 2024-05-02T00:05:48Z gs://k8s-metrics/flakes-latest.json
5 2024-05-02T00:04:09Z gs://k8s-metrics/job-flakes-latest.json
376585 2024-05-02T00:05:08Z gs://k8s-metrics/job-health-latest.json
3 2024-05-02T00:05:20Z gs://k8s-metrics/pr-consistency-latest.json
83496 2024-05-02T00:04:36Z gs://k8s-metrics/presubmit-health-latest.json
3 2024-05-02T00:06:01Z gs://k8s-metrics/weekly-consistency-latest.json
gs://k8s-metrics/build-stats/
gs://k8s-metrics/failures/
gs://k8s-metrics/flakes-daily/
gs://k8s-metrics/flakes/
gs://k8s-metrics/istio-job-flakes/
gs://k8s-metrics/job-flakes/
gs://k8s-metrics/job-health/
gs://k8s-metrics/pr-consistency/
gs://k8s-metrics/presubmit-health/
gs://k8s-metrics/weekly-consistency/
TOTAL: 9 objects, 777494 bytes (759.27 KiB)
We can turn down the old cluster early next week @BenTheElder
SGTM. At some point I'd like to turn down the bigquery datasets and anything else lingering in that project as well.