Unable to deploy on k8s 1.25 - HorizontalPodAutoscaler and PodDisruptionBudget using removed versions

[mila-rodriguez-netapp]

Welcome!

• [X] Yes, I've searched similar issues on GitHub and didn't find any.
• [X] Yes, I've searched similar issues on the Traefik community forum and didn't find any.

What version of the Traefik's Helm Chart are you using?

10.24.0

What version of Traefik are you using?

2.8.3

What did you do?

Attempted to deploy helm chart on a Kubernetes 1.25 cluster with horizontal pod autoscaling and pod disruption budget enabled.

Relevant snippet from values.yaml:

podDisruptionBudget:
  enabled: true
  maxUnavailable: 1

autoscaling:
  enabled: true
  minReplicas: 2
  maxReplicas: 5
  metrics:
  - type: Resource
    resource:
      name: cpu
      targetAverageUtilization: 60
  - type: Resource
    resource:
      name: memory
      targetAverageUtilization: 60

What did you see instead?

unable to recognize \"\": no matches for kind \"PodDisruptionBudget\" in version \"policy/v1beta1\", unable to recognize \"\": no matches for kind \"HorizontalPodAutoscaler\" in version \"autoscaling/v2beta1\"

What is your environment & configuration?

This can be reproduced on any Kubernetes 1.25 environment. I specifically reproduced it on Docker-Desktop with k8s 1.25, and also with minikube.

Additional Information

* `PodDisruptionBudget` in `policy/v1beta1` has been deprecated for a long time and removed in k8s 1.25
  ** Replacement is version `policy/v1` available since k8s 1.21
* `HorizontalPodAutoscaler` in `autoscaling/v2beta1` has similarly been deprecated for a while and removed in k8s 1.25
  ** Note v2beta2 also deprecated and similarly will be removed in 1.26
  ** Replacement is `autoscaling/v2` available since 1.23

[mila-rodriguez-netapp]

The HPA issue is still present in master. Looks like the PDB issue was fixed in 10.24.1.

[mila-rodriguez-netapp]

For what it's worth, we dealt with this locally with a wrapped chart and the following template for HPA:

{{- if .Capabilities.APIVersions.Has "autoscaling/v2/HorizontalPodAutoscaler" }}
apiVersion: autoscaling/v2
{{- else }}
apiVersion: autoscaling/v2beta2
{{- end }}
kind: HorizontalPodAutoscaler
metadata:
  name: {{ template "traefik.fullname" . }}
  labels:
    app.kubernetes.io/name: {{ template "traefik.name" . }}
    helm.sh/chart: {{ template "traefik.chart" . }}
    app.kubernetes.io/managed-by: {{ .Release.Service }}
    app.kubernetes.io/instance: {{ .Release.Name }}
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: {{ template "traefik.fullname" . }}
  minReplicas: {{ .Values.traefik.autoscaling.minReplicas }}
  maxReplicas: {{ .Values.traefik.autoscaling.maxReplicas }}
  metrics:
{{ toYaml .Values.traefik.autoscaling.metrics | indent 4 }}

This uses the current version (autoscaling/v2) with a fallback to autoscaling/v2beta2, which was added in k8s 1.18. We dropped support entirely for v2beta1 because it is so much older (1.16) and because it's not backwards compatible -- the formatting of the metrics changed.

With the above template, we could use the following values (slightly tweaked from the original v2beta1-compatible config I posted in the original bug description):

autoscaling:
  # enabled: true
  minReplicas: 2
  maxReplicas: 5
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 60
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 60

[mloiseleur]

With PR #518 , HPA is now updated to current version.

[FelipeEmerim]

We tried to deploy the latest version and got this error:

Error: UPGRADE FAILED: error validating "": error validating data: ValidationError(HorizontalPodAutoscaler.spec): unknown field "behaviour" in io.k8s.api.autoscaling.v2.HorizontalPodAutoscalerSpec

These are the values we used:

autoscaling:
  enabled: true
  minReplicas: 2
  maxReplicas: 5
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 75
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 75
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Pods
        value: 1
        periodSeconds: 60

I believe the error happens because of this line

[mloiseleur]

Thanks for your detailed report @FelipeEmerim => v12.0.1 should fix this typo.

[FelipeEmerim]

@mloiseleur Worked like a charm! Thank you!

As for this issue, I don't see the warnings about the deprecated autoscaling API anymore. (k8s 1.23). I believe it is working correctly.