feat(nodeadm): enable configuring maxPods with an adjustable offset

[vflaux]

What would you like to be added: I would like to have an option in the node config to adjust the number of maximum pods base on the calculated value. Example:

apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
  kubelet:
    config:
      maxPodsOffset: 1

This would increase the calculated maxPods by 1.

Why is this needed:

When you deploy daemonsets using host network, this do not consume a new IP on nodes. We may want to account for this and increase the maximum number of pods allowed on a node, since the limit is typically based on the number of available IPs on the instance. You can't simply set the maxPods value because you don't always know in advance which instance type will be launched (for example, when using Karpenter).

additional info We currently do this using a userData script:

#!/bin/sh -e
cd /etc/kubernetes/kubelet/config.json.d
jq <00-nodeadm.conf >10-maxpods.conf '{"apiVersion": "kubelet.config.k8s.io/v1beta1", "kind": "KubeletConfiguration", "maxPods": (.maxPods+1)}'

[cartermckinnon]

@vflaux I'll discuss this with the team, sounds like a reasonable + useful addition.

[mselim00]

Support for this will be added in #2446 which supports custom expressions. If you know 3 host networking pods will run on the node (including e.g. the VPC CNI and kube-proxy), you can use an expression like max_pods + 1 to indicate the additional pod, since there's already 2 accounted for in that value.