systemd
systemd copied to clipboard
systemd
firstboot: remove "factory" `{vconsole,locale}.conf` and handle the creation of these in `systemd-firstboot.service`
This moves the handling of the mentioned file to the sd-firstboot service, to allow users to configure these on a per system basis.
As "factory" files these could be modified by system administrators, but due to the entries being commented out they had to be modified to provide any functionality. At that point a system administrator can provide these files themselves with their own tmpfiles.d entry.
I am unsure if I missed anything else that needs to be update.
Closes #31147
I'm not convinced. First of all, these files are also consumed by other programs, especially locale.conf. Secondly, having them in /etc/, even effectively empty, allows sysadmins to easily inspect the build-time default.
First of all, these files are also consumed by other programs, especially locale.conf.
I see what you mean and I asked in the matrix room about if there was a specific reason these 2 files where handled by copying them from factory/etc instead of being written by sd-firstboot during first boot and @poettering did say that it was somewhat contradictory.
The main reason I asked this to begin with is because especially setting keyboard layout during first boot before asking for a root password is somewhat needed to reduce friction/errors during the install process.
Secondly, having them in /etc/, even effectively empty, allows sysadmins to easily inspect the build-time default.
This does make sense. But these files are copied to /usr/share/factory/etc/ compared to the normal /etc/. I have no real problem having them installed to /etc/ like other "default empty configs" that systemd ships, but having them installed to /usr/share/factory/etc/ by default is a bit much when only shipping a /usr/. But I am not 100% sure if there is a reason to do it via tmpfiles instead of with sd-firstboot.
I can rework this if this really even makes sense, else I would just close this PR and issue if there is a specific reason that it's done this way currently.
So this was added in f129d0e77c4c9a0e12ae38cd241cd49846844a80 and 623370e643e228449f5e49c0d82e6e423ae0e0f7.
With a very specific usecase each time. What do we do about that?
So I am a bit unsure about this. I can see both sides. It's not clear to me at all what is the right approach here.
I didn't see this PR and I arrived at the conclusion that those files should be removed. My commits it https://github.com/systemd/systemd/pull/31225 had more discussion why those files are not necessary.
Essentially, the factory directory is supposed to provide files without which the system is not functional. I.e. it's mostly a crutch for programs which only look in /etc. Another use case might thing like /etc/issue, without which the system is functional, but the system works nicer with them. The factory dir should not be used for files that document defaults or fulfil other cosmetic use cases. It certainly doesn't make sense to add files to factory until we end up with "We have empty /etc, but actually it's all now in /usr/share/factory/etc, so the end system looks exactly the same".
General comment: please make sure that the actual commits that are merged describe the changes fully. The discussion in the PR is not going to become part of the git history.
I'll reword the commits.
To just have these files generated as defaults in /etc/ without them being in factory like the rest of the defaults files, where would I need to add them? I would guess src/core, but I just want to double check. (Should I even add that?)
So this basically means that SYSTEMD_DEFAULT_KEYMAP and SYSTEMD_DEFAULT_LOCALE lose their meaning though?
So this basically means that SYSTEMD_DEFAULT_KEYMAP and SYSTEMD_DEFAULT_LOCALE lose their meaning though?
I may be misunderstanding the question, but both are still used in the code as fallback when no external option sets them. That still allows to change the default locale/keymap at build time instead of always defaulting to us and C.UTF-8.
https://github.com/systemd/systemd/blob/e374109efb582f3f3a417fd3f8c71121873c7b0c/src/vconsole/vconsole-setup.c#L106 https://github.com/systemd/systemd/blob/9182658d3b98803e52c83756f082d19ee4c4e50e/src/shared/locale-setup.c#L276
SYSTEMD_DEFAULT_LOCALE existed before the factory files were introduced. The rationale described in commit msg clearly stated this.
If I assume correctly, the commit that allows the sd-firstboot service to ask for all its supported options is not a problem, did I understand that correctly? And if so, I could split that out to a separate PR.
The vconsole.conf file should also not really have any impact, considering the file even currently is entirely commented out. Removing this file from /usr/share/factory/etc/ is my main "goal", as that would allow setting the default keymap before needing to enter a password when booting with an empty /etc/ and the modified service file.
The problem with locale.conf if I understood it correctly is, that programs/scripts might unconditionally read the file directly and always expect at least LANG to be set to determine their locale. So that file needs to always exist after boot, even if it was previously deleted (and is copied again on next boot). Did I understand that correctly?
I have no problem re-adding both files to just /etc/ to indicate the defaults and preserve the status quo for systems expecting/shipping a populated /etc/ (I would just need to know where I should put the source files). It's just both these files can/should be written during first boot by the sd-firstboot service when only shipping a generic /usr/ partition, as each user might want something different (even I use different keymaps on different PCs).
Would shipping these files just in /etc/{vconsole,locale}.conf (and not in /usr/share/factory/) address your concerns @YHNdnzj or is my assumption/understanding of your objections in paragraph 3 correct?
Would shipping these files just in /etc/{vconsole,locale}.conf (and not in /usr/share/factory/) address your concerns @YHNdnzj or is my assumption/understanding of your objections in paragraph 3 correct?
Hmm, yeah, maybe this is the way to go. So that systems with /etc/ already populated still get those files.
I have now changed the first commit to move them out of the factory/templates dir to simply a templates dir since I was unsure where else to put them.
As a note: when building with mkosi I've been getting the error cp: not writing through dangling symlink '{mkosi-workspace}/root/etc/vconsole.conf' (the file links to default/keyboard). Unsure why this is happing as when building/installing normally vai meson everything works fine.
The problem with
locale.confif I understood it correctly is, that programs/scripts might unconditionally read the file directly and always expect at leastLANGto be set to determine their locale. So that file needs to always exist after boot, even if it was previously deleted (and is copied again on next boot). Did I understand that correctly?
Was this described before somewhere? And also, is this really true? Systemd will set LANG and other settings for all children, so pretty much anything already has the environment set up. What are those consumers?
I don't know about the exact case I described, but I do know/assume that it isn't too unlikely that some install scripts or so expect the files to at least be in /etc/ at the time they run, which might be before an actual boot (new system was installed to /mnt/ and now they want to set LANG with sed).
This case does need these file to be in /etc/ but they aren't required to be in the factory dir. And any program hard expecting the files to be in /etc/ are still broken should these files be manually removed and I doubt they really expect that these files would be recreated every boot.
so what about this approach to retain the reason why 623370e643e228449f5e49c0d82e6e423ae0e0f7 was added in the first place:
in systemd-firstboot, if SYSTEMD_DEFAULT_KEYMAP or SYSTEMD_DEFAULT_LOCALE are defined at build time, but --prompt-keymap/--prompt-locale are not specified, and no credential is set for these two fields, then write out the two files with these contents?
This would effectively mean that the initial population of these files is just migrated from tmpfile.d's factory logic into firstbot, but it would be retained in its behaviour otherwise, but not conflict with firstboot's own prompting logic anymore.
I mean, I sympathize with @lucab's original reasoning here: there are multiple consumers of these files, it's not a private config file of ours, hence it probably makes sense to populateit with our defaults to avoid any ambiguity on what to fall back to if the files don't exist.
Does that make sense?
in systemd-firstboot, if SYSTEMD_DEFAULT_KEYMAP or SYSTEMD_DEFAULT_LOCALE are defined at build time, but --prompt-keymap/--prompt-locale are not specified, and no credential is set for these two fields, then write out the two files with these contents?
Hmm, I personally don't use firstboot. And a typical Arch installation comes with an already populated /etc/. I still want these files to be present in such setups.
Though having firstboot write out these files when /etc/ is not populated also makes sense... I don't know if we should do both
This would effectively mean that the initial population of these files is just migrated from tmpfile.d's factory logic into firstbot, but it would be retained in its behaviour otherwise, but not conflict with firstboot's own prompting logic anymore.
This is basically what the second commit does, just instead of doing that special logic it just prompts for all options. firstboot prompts only for things for which the files don't exist, the only thing missing is having firstboot default to SYSTEMD_DEFAULT_KEYMAP and SYSTEMD_DEFAULT_LOCALE when nothing is entered in the prompt as it currently skips/doesn't write anything when entering nothing in the prompt.
So where are we with this now? the proposal i made does that work for everyone?
I don't know if it's really necessary to make the logic that convoluted. I think simply having the locale/keymap default to their respective configured default when --prompt-{locale,keymap} is passed and nothing is entered in their respective prompt should be enough (although I am not a big fan of that specifically and I think prompting the user to select one is enough, if they explicitly don't want to set one when asked it's kinda on them).
But other than that I would be fine with it
This would effectively mean that the initial population of these files is just migrated from tmpfile.d's factory logic into firstbot, but it would be retained in its behaviour otherwise, but not conflict with firstboot's own prompting logic anymore.
I like this general idea.
in systemd-firstboot, if SYSTEMD_DEFAULT_KEYMAP or SYSTEMD_DEFAULT_LOCALE are defined at build time, but --prompt-keymap/--prompt-locale are not specified, and no credential is set for these two fields, then write out the two files with these contents?
Sounds fine too, with the caveat that I think we should actually add --prompt-keymap/--prompt-locale to the systemd-firstboot.service as is done in @NekkoDroid's patch. This would mean that there is a slight change in behaviour.
I mean, I sympathize with @lucab's original reasoning here: there are multiple consumers of these files, it's not a private config file of ours, hence it probably makes sense to populateit with our defaults to avoid any ambiguity on what to fall back to if the files don't exist.
OK. Re-reading the original commit, the justification is a bit vague, i.e. those consumers are not named, but I think we can assume that there are some. Both files are widely known and @lucab wouldn't be making the patch if it wasn't useful for something.
I don't know if it's really necessary to make the logic that convoluted. I think simply having the locale/keymap default to their respective configured default when
--prompt-{locale,keymap}is passed and nothing is entered in their respective prompt should be enough (although I am not a big fan of that specifically and I think prompting the user to select one is enough, if they explicitly don't want to set one when asked it's kinda on them).
I think doing both is fine: write out the default to file as @poettering suggested if not asked or configured via credentials, suggest the default if prompted.
About asking for both: I think we should ask, but we should suggest a sensible default. Currently each question is "independent", but that is just not user-friendly. If a locale is entered, then almost certainly the user will want the mapping that matches the locale. E.g. if LANG=de_DE.UTF-8, then "de" should be provided as the default keymap. Then the user can just press Enter to accept the default. If the whole system worked in a way where you enter the locale and then you can "blow through" the other questions by pressing enter a few times, and actually get a sensibly configured system, then having an additional question is not a problem. But right now you have to answer in full every question for which the compiled-in default is not suitable, and that is rather annoying. But I wouldn't make this a prerequisite for this pull request. Let's get the first part of "moving the logic from tmpfiles to firstboot" merged, and then we can work on improvements to firstboot.
Re: improvements to firstboot:
- In cases where we want the default to be configured in a file on the system (LANG=, KEYMAP=), provide the compile-time default as the default in prompt.
- Make the default for KEYMAP depend on the answer for LANG.
- Make the default for timezone depend on the answer for LANG. This one is more tricky, because there are locales where the countries using those locales are spread over multiple time zones. But OTOH, there are plenty of locales which almost always are used in one timezone. We could make the UX much nicer at least for those countries.
- Firstboot know the list of valid locales, but accepts any answer, which doesn't make any sense, esp. considering how tricky it is to write out strings like "ks_IN.UTF-8@devanagari" from memory. Thus, if given a partial answer, we should give a smaller list as a hint. Maybe something like "'ks' is not a known locale name. Here is a list of similar locale names: 1) ks_IN.UTF-8, 2) ks_IN.UTF-8@devanagari", and then repeat the prompt. And provide the first matching option as the default that can be accepted with just Enter. This would work nicely for locales like "pl_PL.UTF_8", where there is just one locale matching the prefix "pl".
- Maybe we could also implement tab completion? That'd be a nice enhancement, but I don't know how hard that'd be.
- Since we provide default answers and the empty answer (Enter) cannot be used to skip an answer, allow "-" as an answer meaning "skip". The prompt would need to be adjusted, or maybe the introductory message.
- Stop saying "No data entered, skipping." at warning level. In particular in interactive mode this is not useful, the user knows that they pressed Enter.
Some of those are easy, some other hard…
Just in order to add some context, the original issue that users were hitting was https://github.com/coreos/fedora-coreos-tracker/issues/568.
The consumers may vary from a TTY shell to a SSH session without LANG forwarding, and possibly others.
If those config files are missing, different tools will use their own built-in defaults.
I'm all for having systemd-firstboot customize those; the tmpfiles fragment covers the case when such customization does not happen.
I'm all for having
systemd-firstbootcustomize those; the tmpfiles fragment covers the case when such customization does not happen.
Hmm, but that means that to not regress for coreos, which doesn't use firstboot, we'd need to keep the tmpfiles config.
Hmm, but that means that to not regress for coreos, which doesn't use firstboot, we'd need to keep the tmpfiles config.
That would then somewhat defeat the purpose of this, since tmpfiles are copied before firstboot is run, preventing firstboot from setting the values.
I don't have much experience with CoreOS (always had some problem during installation), does it ship with an empty /etc/? and if so, would it be an option to add the 2 files + a custom tmpfiles.d entry on the distro/image side? If it does ship a populated /etc/ I might just be missing the issue, as how it currently stands the files are still present in /etc/. Or is there some other way this problem could be addressed?
Regarding actually implementing the logic: I don't know if I am currently proficient enought in C-isms to implement it, nor do I have a lot of time at the moment so it might take a bit until I can actually sit down to do it.