Bottlerocket settings operator

[etungsten]

What I'd like: A Bottlerocket settings operator that can apply a set of bottlerocket API settings across multiple nodes in a cluster at the same time - A new way to configure multiple bottlerocket nodes declaratively.

Any alternatives you've considered: Manual configuration is cumbersome when managing a substantial number of nodes.

[tjkirch]

We had talked about using SSM (or similar) to send API requests to fleets. SSM makes that easy.

I think it'd help to expand on why an operator would be a good fit here, because they have a lot more overhead.

[rverma-jm]

I would prefer SSM over Operator for managing any host's related stuff. It simplifies access control and security easily. Also I believe with the recommended PSP, getting elevated access from container won't be possible.

[samuelkarp]

Both SSM and an operator can make it easy to drive one-off API requests to Bottlerocket nodes in your cluster. However, one of the things that's easier with an operator as compared to SSM is to model it as a control loop and drive consistent settings in a cluster. An operator can periodically query the API and adjust it when the observed settings are different from the desired settings; drift detection and correction. This makes it possible to add new nodes to your cluster and have them configured with your desired settings automatically, without having to modify user data in a launch template or otherwise change how new nodes are launched.

[bcressey]

Suggested by @cmanzi in #987:

I wonder if the config could even be managed in a ConfigMap that the update-operator watches? Being able to manage node configs from a K8s resource that can be controlled in a gitops repo somewhere would be fantastic.

[springroll12]

Any progress on this operator? It would be fantastic to be able to control settings from a central location.

[zmrow]

@springroll12 We don't have anything new to report on this, but will update the issue and it's tags if and when we prioritize it!