k8s.io icon indicating copy to clipboard operation
k8s.io copied to clipboard

`Quota 'CPUS' exceeded` error on containerd presubmits

Open akhilerm opened this issue 1 year ago • 5 comments

The pull-containerd-node-e2e presubmit is failing with the following error

ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Quota 'CPUS' exceeded.  Limit: 100.0 in region us-central1.
	metric name = compute.googleapis.com/cpus
	limit name = CPUS-per-project-region
	limit = 100.0
	dimensions = region: us-central1
Try your request in another zone, or view documentation on how to increase quotas: https://cloud.google.com/compute/quotas.

Ref: https://testgrid.k8s.io/containerd-presubmits#pull-containerd-node-e2e

Error log: https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/directory/pull-containerd-node-e2e/1754593017232101376#1:build-log.txt%3A1168

The following project is used for the containerd presubmit

gcp-project=cri-c8d-pr-node-e2e
gcp-zone=us-central1-f

/cc @ameukam

akhilerm avatar Feb 05 '24 20:02 akhilerm

/kind bug /area infra /priority backlog

ameukam avatar Feb 06 '24 08:02 ameukam

The SIG don't currently have access to the projects used by Containerd so we don't have a way to address this quickly other than escalate to our Google contacts. May take a while to solve this.

ameukam avatar Feb 06 '24 08:02 ameukam

cc @dims @BenTheElder

ameukam avatar Feb 06 '24 08:02 ameukam

@ameukam let's switch the containerd jobs to boskos?

dims avatar Feb 06 '24 12:02 dims

+1, let's phase out using a google.com project here and use community boskos

BenTheElder avatar Feb 06 '24 23:02 BenTheElder

@akhilerm are we done with this ? (Phase out to community boskos should be a different issue)

ameukam avatar Apr 18 '24 07:04 ameukam

Yes, for now this is done.

Will have a separate issue for phasing to community infra.

akhilerm avatar Apr 23 '24 14:04 akhilerm