flatcar
Provide a Flatcar Tools, Extra-Tools, and Dev Extension
Current situation
For building kernel modules or debug symbols, the devcontainer in systemd-nspawn (or the third-party Docker images) has to be used.
For new package requests we have to normally say no and point at the bakery. Existing tools such as strace, tcpdump, or git are on the base image even if most users don't need them.
Impact
The UX with the second environment that's persisted to disk and only has access to some host paths is not the best.
We are stuck with not really adding new tools because of image size concerns.
Ideal future situation
Build developer container packages into a systemd-sysext image (gcc, gdb) and publish as release artifact and as signed update payload for consumption as official Flatcar extension. It gets activated through a line with NAME in /etc/flatcar/enabled-sysext.conf.
We can limit the scope to gcc and gdb for kernel modules and debugging or we can include emerge for more.
A new "tools" extension enabled by default (git, strace, tcpdump, vim(-full) go there from the base image) and an optional "extra-tools" extension (nano, htop, tmux can finally be provided to users).
Implementation options
Call it Flatcar "dev", Flatcar "devtools", or Flatcar "compiler" extension. Consider to provide debug symbols through the HTTP fetch mechanism that gdb now supports. Otherwise the download size is too bloated.
Additional information
Depending on when the "extra" extension gets implemented, make sure that there is no conflict in files they provide. If the "dev" extension requires tools from "extra", activation of "extra" will be implied by "dev".
Note that for debugging with gdb the current library path is something like /build/ARCH but it should become the host path.
About the tools extension that would have stuff such as vim: strace could also go into this new tools extension. New things like htop would then go to something like extra-tools. The extra-tools extension should depend on tools so that they don't end up shipping the same library files if there are common dependencies. Compiler and debugger go into the dev extension and this depends on both tools and extra-tools.