Breaking changes in 2.7.0 with hpa metrics and Kubernetes 1.22

[dankolt]

After merge #1906 hpa metrics can't collect from Kubernetes 1.22. Kube-state-metrics logs tells that object *v2 HorizontalPodAutoscaler is undefined. If it's expected behavior, then I guess need some note in release notes that minimal support k8s version is 1.23.

[razum2um]

Confirm this issue, k8s-1.22, prometheus-operator-kube-state-metrics-2.7.0 logs:

W0109 13:24:32.432243       1 reflector.go:424] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:169: failed to list *v2.HorizontalPodAutoscaler: the server could not find the requested resource
E0109 13:24:32.432271       1 reflector.go:140] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:169: Failed to watch *v2.HorizontalPodAutoscaler: failed to list *v2.HorizontalPodAutoscaler: the server could not find the requested resource

so for 1.22 the proper kube-state-metrics version is 2.6.0

p.s. and the maximum compatible chart of prometheus-operator is kube-prometheus-stack-42.0.3

[mrueg]

I assume v2beta2 support got dropped and v2 got introduced in v1.23 (https://kubernetes.io/blog/2021/12/07/kubernetes-1-23-release-announcement/#horizontalpodautoscaler-v2-graduates-to-ga).

See: https://github.com/kubernetes/kube-state-metrics/pull/1906

https://github.com/kubernetes/kube-state-metrics#compatibility-matrix

Since kubernetes 1.22 has already reached EOL upstream (https://kubernetes.io/releases/patch-releases/#non-active-branch-history), I would suggest to upgrade your clusters.

[razum2um]

Agree on upgrading, but worth mentioning a lot of people running in the cloud, in particular in aws/eks, where EOLs lagging behind by ~2 upstream versions, i.e. 1.22 is EOL in 3 months and released just 9 months ago

[mrueg]

I can see your point, and you might want to talk to your public cloud provider to help out here. If cloud vendors support EOL versions of Kubernetes for a longer time, they are more than welcome to contribute to the project.

[logicalhan]

/triage accepted /assign @mrueg

[k8s-triage-robot]

This issue has not been updated in over 1 year, and should be re-triaged.

You can:

• Confirm that this issue is still relevant with /triage accepted (org members only)
• Close this issue with /close

For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/

/remove-triage accepted

[dashpole]

1.22 is ancient at this point, so this probably isn't relevant anymore

/close

[k8s-ci-robot]

@dashpole: Closing this issue.

In response to this:

1.22 is ancient at this point, so this probably isn't relevant anymore

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.