Support CPUManager policy options

[jbg]

What I'd like:

I'm able to set the static CPU Manager policy with settings.kubernetes.cpu-manager-policy, but I can't set any options for it. I'd like to be able to set the full-pcpus-only policy option.

Any alternatives you've considered:

Not aware of any.

[cbgbt]

Hi, thanks for the request. I can see that the additional options for static CPU Manager policies are currently feature gated (docs).

Looking at the kubelet configuration documentation, it seems like the policy options are specified as a map in a separate field, cpuManagerPolicyOptions. We'll have to decide for Bottlerocket settings if we want to follow suite or provide the settings as a composite type (by expanding cpu-manager-policy to allow a shape which includes both the policy as well as its options)

Following the same pattern is probably least surprising, though cpuManagerPolicy and its options have interconnected validity rules, and Bottlerocket's settings implementation doesn't offer many tools for creating validation rules which span multiple settings models.