bottlerocket icon indicating copy to clipboard operation
bottlerocket copied to clipboard

support automated reconciliation of kernel command line settings

Open bcressey opened this issue 3 years ago • 5 comments

What I'd like: I'd like a way to provide kernel or init command line parameters via user data or bootstrap container, and have the node automatically reboot into a state where they're applied.

This can be optional, and off by default.

An example use case would be enabling cgroup v2:

[settings.boot]
reboot-to-reconcile = true

[settings.boot.init-parameters]
"systemd.unified_cgroup_hierarchy" = "1"

Any alternatives you've considered: Handling the reboot in a bootstrap container, but this is potentially awkward if I have multiple bootstrap containers, some of which apply settings and one of which issues the reboot, since there is no explicit ordering among them.

bcressey avatar May 24 '22 17:05 bcressey

Implementation thoughts:

We might get away with ordering constraints on the bootstrap container service units, such that they were all RequiredBy and Before the reconciliation unit. Otherwise we could use a new reconciled.target to guarantee this. The important thing is that we issue any required reboot before we start multi-user.target.

I don't recall whether parameters passed by bootconfig are de-duped against parameters on the kernel command line. If not, then a simple comparison of /proc/bootconfig (current state) to serialized form of settings.boot could suffice. Otherwise it's more difficult.

It seems OK to ignore the opposite case - when a setting is not in bootconfig but is on the default kernel command line - and just accept that we'll duplicate the parameter and take an unnecessary reboot to sync things up.

bcressey avatar May 24 '22 17:05 bcressey

@etungsten, @zmrow, @arnaldo2792 - I'd like to get your thoughts too.

bcressey avatar May 24 '22 17:05 bcressey

This sounds like a good idea. I agree the current behavior with settings.boot is not ideal where the user has to manually reboot the host to apply boot settings specified in user-data.

I don't recall whether parameters passed by bootconfig are de-duped against parameters on the kernel command line. If not, then a simple comparison of /proc/bootconfig (current state) to serialized form of settings.boot could suffice. Otherwise it's more difficult.

Parameters passed by bootconfig are not de-duped against parameters on the kernel cmdline. You can specify the same parameters multiple times. In general, earlier specifications are overwritten by later ones. Bootconfig parameters are prepended so parameters specified directly the to kernel cmdline always take precedence. See here.

etungsten avatar May 31 '22 21:05 etungsten

All of this sounds reasonable to me!

zmrow avatar Jun 06 '22 18:06 zmrow

We might get away with ordering constraints on the bootstrap container service units, such that they were all RequiredBy and Before the reconciliation unit.

I like the idea, and I think this should work, I wonder if we should extend prairiedog to evaluate if a reboot is required.

arnaldo2792 avatar Jun 14 '22 17:06 arnaldo2792