Resource limits forbid running of the syncer

[Smidra]

What happened?

• I downloaded my values.yaml from GitHub
• I enabled quotas at "policies: > resourceQuota: > enabled: true"
• I created the vCluster with helm as follows helm upgrade --install vcluster-r05 vcluster --version 0.20.0-beta.2 --values vcluster.yaml --repo https://charts.loft.sh --namespace vcluster-r05 --create-namespace --repository-config='' --wait --wait-for-jobs
• The syncer is not created. Stateful set has 0/1 pods ready. The events state that: create Pod vcluster-r05-0 in StatefulSet vcluster-r05 failed error: pods "vcluster-r05-0" is forbidden: failed quota: vc-vcluster-r05: must specify limits.cpu for: syncer

What did you expect to happen?

• The vCluster is created without a problem.

How can we reproduce it (as minimally and precisely as possible)?

See above.

Anything else we need to know?

It can be a good idea to set CPU requests and not set CPU limits. With vCluster, we do that in the syncer limits (controlPlane: > statefulSet: > resources: > limits:) definition. In the values.yaml there should also be a way to disable cpu limits.

What do you think about making the CPU unlimited by default?

Host cluster Kubernetes version

$ kubectl version
Client Version: v1.29.3
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.6

Host cluster Kubernetes distribution

default - k8s

vlcuster version

$ vcluster --version
# paste output here

Vcluster Kubernetes distribution(k3s(default)), k8s, k0s)

Default - k8s

OS and Arch

OS: Ubuntu
Arch: x86

[heiko-braun]

Thanks for reporting this. I will discuss it within team.

[rohantmp]

This is a kubernetes limitation, not a vcluster one:

If quota is enabled in a namespace for compute resources like cpu and memory, users must specify requests or limits for those values; otherwise, the quota system may reject pod creation. Hint: Use the LimitRanger admission controller to force defaults for pods that make no compute resource requirements.

( https://kubernetes.io/docs/concepts/policy/resource-quotas/ )

[rohantmp]

Ah's an issue that we don't have a default cpu limit, so enabling resource quotas without writing in an cpu.limit doesn't work

[rohantmp]

Like you said, I don't think it's a great idea to have a default cpu limit, so I think this is better left alone for now, but will see about automatically adding one if resourceQuota is enabled

[Smidra]

If the CPU limit would enable only when the Resource Quota is enabled it would be a wonderful solution. Good idea @rohantmp

[FabianKramm]

Hey @Smidra ! Is there a reason you enabled resource quota but not the limit range via policies.limitRange.enabled, because this should set the missing cpu automatically.

[Smidra]

Hello @FabianKramm, you are correct that enabling LimitRange fixes this problem. :+1:

The reason for my complaint is that "Limit Range" is disabled by default and there are no syncher CPU limits by default. If you just enable quotas, than vcluster will "mysteriously" hang because StatefulSet will become stuck.

In my opinion this is a strange default behavior. If you decide not to change it, we should at least adress it in the comment of the values file or in the documentation.