operator
operator copied to clipboard
VMAgent doesn't work with VPA
Hello, we are trying to setup a VPA on top of a VMAgent. Unfortunately we get the following error from the VPA
E0718 13:54:51.559558 1 cluster_feeder.go:570] Cannot get target selector from VPA's targetRef. Reason: Unhandled targetRef operator.victoriametrics.com/v1beta1 / VMAgent / application, last error Resource victoria/application has an empty selector for scale sub-resource
Here the manifest of the VPA:
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: vmagent-application-vpa
namespace: victoria
spec:
targetRef:
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAgent
name: application
updatePolicy:
updateMode: "Auto"
resourcePolicy:
containerPolicies:
- containerName: vmagent
maxAllowed:
cpu: 1
Looking at the CRD of the VMAgent I would expect it to work since
subresources:
scale:
labelSelectorPath: .status.selector
specReplicasPath: .spec.shardCount
statusReplicasPath: .status.shards
status: {}
is configured.
Adding another piece to the CRD
- additionalPrinterColumns:
- description: current selector
jsonPath: .status.selector
name: Selector
type: string
shows an empty string.
k get vmagent application -n victoria
NAME SHARDS COUNT REPLICA COUNT SELECTOR
application 0 1
Any help is appreciated. Maybe we just do something wrong?
Hello!
Looks like vpa has some conflict with operator on scale subresource, so it's not working now.
What's your case to scale vmagent? Maybe horizontal scaling can help.
The issue at vpa is from us as well. We can't pinpoint the root cause of this issue yet. We have a fleet of EKS Clusters with the same configuration for metrics. Each cluster has different workloads so the VMAgent has different resource requirements. The compute resources are scaled very dynamically in dependency of the consumption. Therefore the VPA.
Oh, looks like operator needs to change something here to fit vpa then.
Hello! Are there any updates on this? seems to effect us
We also have an issue with this as we have policies in place which checks for existence of a VPA but we cant define a working one.
Sorry for delay. I think, we should prioritize it.
Fix was added with linked PR. It'll be a part of the next release.
Also, I think, that we should add direct support for VPA and HPA resources as .spec field for:
vmalert- VPAvmagent- VPA+HPAvmcluster- VPA+HPA for select and insert componentsvmauth- HPA + VPAvmalertmanager- VPA.
Operator is responsible for setting targetRef field with corresponding value for reference (deployment/statefulset).