karpenter icon indicating copy to clipboard operation
karpenter copied to clipboard

Published 20 hours ago • verified publisher icon kubernetes-sigs

Support new topologySpread scheduling constraints

Open tzneal opened this issue 1 year ago • 11 comments

Description

Observed Behavior:

Karpenter doesn't support:

  • matchLabelKeys
  • nodeAffinityPolicy
  • nodeTaintsPolicy

These fields were introduced into beta in 1.27.

Expected Behavior:

Reproduction Steps (Please include YAML):

Versions:

  • Chart Version:
  • Kubernetes Version (kubectl version):
  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

tzneal avatar Aug 01 '23 13:08 tzneal

I'll work on this issue

sadath-12 avatar Oct 03 '23 12:10 sadath-12

/assign

jmdeal avatar Dec 06 '23 21:12 jmdeal

/remove-help

jmdeal avatar Dec 06 '23 21:12 jmdeal

What is current behaviour of the karpenter for nodeAffinityPolicy?

ilkinmammadzada avatar Feb 06 '24 17:02 ilkinmammadzada

Currently Karpenter doesn't recognize either nodeAffinityPolicy, so it will always be treated as honor. Originally the linked PR was blocked on some 1.29 features and I haven't had the time to do the performance testing since we got unblocked on 1.29 testing. Those features have since been scoped out but the linked PR will add support for nodeAffinityPolicy, nodeTaintPolicy, and matchLabelKeys.

jmdeal avatar Feb 06 '24 17:02 jmdeal

Hey everyone! Thanks for putting in the effort to support these new constraints! We just hit this exact issue in our clusters, where we are using matchLabelKeys together with topologySpreadConstraints to spread our pods across AZs.

I was wondering if you currently have a timeline for getting this PR merged? It seems the code is already ready for some time and it is just waiting for a rebase/performance test?

dominik-dezordo-vc avatar May 03 '24 07:05 dominik-dezordo-vc

This has been backlogged for me for a little while, but I should have some bandwidth to get this wrapped out within the next week. Like you said, it really should only be performance testing and a rebase at this point that's left.

jmdeal avatar May 03 '24 16:05 jmdeal

Amazing, thanks for the fast response. Really appreciate the work you are doing with karpenter!

dominik-dezordo-vc avatar May 03 '24 17:05 dominik-dezordo-vc

We are also really awaiting for matchLabelKeys as now topologySpread is applied to all versions of a deployment which sometimes works not as optimal as we would like.

pznamensky avatar Jun 03 '24 11:06 pznamensky

Hello everyone In my company, we heavily rely on the matchLabelKeys setting and we've also encountered similar issues when new pods of the rollout were in the Pending because Karpenter didn't respect the matchLabelKeys and made the calculations taking into account pods of the previous revision. It caused an incident during the night preventing us from applying hotfixes to the production workloads.

Could you please share the ETA of this feature?

abrazouski avatar Jun 13 '24 10:06 abrazouski

Hi, the lack of matchLabelKeys support, in particular, is also causing problems for us.

Is there anything we can do to help, test a custom build of Karpenter that supports this feature or something?

hamishforbes avatar Aug 14 '24 00:08 hamishforbes