[RFC 0165] Bootspec v2

[RaitoBezarius]

This introduces the second revision of RFC-0125, Bootspec, addressing the feedback we received in https://github.com/NixOS/rfcs/pull/125 and building on our experience of Bootspec v1.

Rendered

TODO:

• [x] Clarify transition period: generate v1 and v2 plus offer synthesis to v1 using v2 and non-bootspec data
  • [x] Mention existing infrastructure tools that will take a NixOS closure and transform it into something else
• [x] Move initrd secrets feature inside its own extension
• [ ] Devicetree discussion
  • [ ] Do we collapse FDTDIR / devicetree into… nothing or just FDTDIR?

[RaitoBezarius]

I would like to particularly invite @samueldr for clear expertise on devicetree and non-x86 systems in Nix(OS) ecosystem (of course, you may not have time, and I am already thankful for the time you provided me online).

I would like @lheckemann to join me again as a co-author if that's possible.

I would like to invite the previous shepherds of bootspec: @govanify @06jellyjac. I would like to invite the people who made a lot of relevant comments which prompted this version 2: @hesiod @amjoseph-nixpkgs (if you have time!)

I would like to invite my colleagues of lanzaboote: @nikstur and @blitz to comment from their perspective as they worked a lot with Bootspec v1, maybe you can even become shepherd this time!

I would like to invite the initial authors of the previous revision: @grahamc and @cole-h — thank you for your support!

And I am probably forgetting a lot of folks, so everyone is welcome if you want to participate in this second revision.

This work is funded by the Tech Sovereign Fund as part of https://github.com/nix-community/projects/blob/main/proposals/nixpkgs-security.md#boot-chain-security (milestone 3), if someone wants to take a chunk of the money working on that with an active role, I would be glad to distribute some in the limits of classical bureaucracy and goal definitions regarding this specification.

[grahamc]

By the way, I like this RFC and I think its going in a good direction 👍 thank you!

[ghost]

if you’d like to pursue amending something decided in an RFC (like where to track and amend the boot spec format) probably a new RFC is the way to go.

This is not how I understand the RFC process to work, nor how it describes itself:

https://github.com/NixOS/rfcs/blob/c8569f6719356009204133cd00d92010889ed56d/rfcs/0001-rfc-process.md?plain=1#L47-L49

RFCs are for "substantial changes" only.

Reinterpreting the RFC process as granting "write-protect unless RFC" to certain files in nixpkgs, covering even minor experimental changes, would make the RFC process even more dysfunctional than it already is.

See also https://github.com/NixOS/rfcs/pull/165#discussion_r1379648995

[RaitoBezarius]

Reinterpreting the RFC process as granting "write-protect unless RFC" to certain files in nixpkgs, covering even minor experimental changes, would make the RFC process even more dysfunctional than it already is.

We are not saying that the RFC process grants write protect unless RFC for minor experimental changes. The extension field offers full flexibility for anyone to do even major experimental changes, we just want to ensure that all the ecosystem is protected by strong stability properties, e.g. breaking changes are strictly forbidden, semantics change in the interpretation of values are highly ill-advised (re: the discussion on nulls).

Maybe what this is all hinting at is that we need another way to track the continuous life of specifications like those, I am just afraid that if we don't go through RFC, this will just push the discussion / work in an obscure place, and we will lose the diversity of people who comes to read this or are involved in this process. Hence, also why I believe that RFC process is the right place.

I would definitely be able to go around and push my Bootspec v2 by myself and try to ping everyone like I did here, but I think it would reduce the discoverability of the whole ecosystem of work that Bootspec offers to anyone stumbling on it.

Now, maybe, we can have the perspective of potential other members of the RFC steering committee on that?

[blitz]

@RaitoBezarius Out of curiosity, how would the initrd secrets interact with encryption? Does the user have to encrypt them or do they have to be around in plaintext at bootloader installation time?

[RaitoBezarius]

@RaitoBezarius Out of curiosity, how would the initrd secrets interact with encryption? Does the user have to encrypt them or do they have to be around in plaintext at bootloader installation time?

I believe they ought to be around in plaintext at bootloader installation time, but I do have this open question on: should we let systemd decrypt the initrd secrets at switch to configuration time and make them available only for this script execution? Thus, we would have decryption happening only at installation by systemd!

[sdht0]

Sharing my thoughts on XBOOTLDR support inside bootspec, as per @RaitoBezarius's comments at https://github.com/NixOS/nixpkgs/pull/260241#issuecomment-1789190031, https://github.com/NixOS/nixpkgs/pull/226692#issuecomment-1742817997 and https://github.com/nix-community/lanzaboote/issues/173#issuecomment-1533406338.

In UEFI systems, the EFI system partition (ESP) might have a small size and not easily resizable (e.g., 100MB on a Windows dual-boot). This results in NixOS errors when trying to store kernels and initrd from multiple generations (e.g., here and here). The solution (apart from trying to resize or relocated the ESP) is use a separate partition for storing the boot files and use the ESP only to install the bootloader.

In NixOS, GRUB already supports this scenario. systemd-boot can also use an extra XBOOTLDR partition, but NixOS does not support configuring such a partition yet (https://github.com/NixOS/nixpkgs/pull/260241).

@RaitoBezarius suggested that XBOOTLDR-type partition should perhaps be explicitly supported inside bootspec as an extension. However, after reading more on bootspec, I'm not sure if XBOOTLDR fits in here. As was pointed out above, bootspec applies to individual generations, while configuring the ESP and XBOOTLDR partitions is a system wide action. Individual generations will not decide which partition its files should be installed to, and enabling the XBOOTLDR partition would mean files from all generations will be moved to that partition. I might be missing something and perhaps @RaitoBezarius can chime in with how bootspec and XBOOTLDR can interact?

Windows unfortunately still suggests the ESP size to be 100MB (even Windows updates fails sometimes because of that), so I think NixOS should generally support the XBOOTLDR partition in it's bootloaders, whether individually in each bootloader or as a policy in something like this bootspec.

[tejing1]

I may be misunderstanding here, but from what I can see, the bootloader backend's responsibility is totally unspecified when it comes to initrd secrets.

Is this proposal assuming that the nix code creating a generation and the bootloader backend are just magically in agreement about this? Does no one ever switch to a different bootloader backend on an existing system with existing generations?

What, precisely, is the bootloader backend required to ensure about the stage1 environment when boot.json contains a non-empty initrdSecrets value? I think this needs to be specified clearly, to the point that, in particular, switching to a different bootloader backend can be done and have previously existing generations actually function on the new backend.

[RaitoBezarius]

I may be misunderstanding here, but from what I can see, the bootloader backend's responsibility is totally unspecified when it comes to initrd secrets.

Is this proposal assuming that the nix code creating a generation and the bootloader backend are just magically in agreement about this? Does no one ever switch to a different bootloader backend on an existing system with existing generations?

What, precisely, is the bootloader backend required to ensure about the stage1 environment when boot.json contains a non-empty initrdSecrets value? I think this needs to be specified clearly, to the point that, in particular, switching to a different bootloader backend can be done and have previously existing generations actually function on the new backend.

We can specify this but I surmise this will be relying a lot on systemd stage 1 semantics anyway.

[JulienMalka]

If there is a need for more shepherds, I'd like to self nominate for that. I've been following this RFC carefully, worked with bootspec v1 implementation in the systemd-boot backend (see https://github.com/NixOS/nixpkgs/pull/263442), and have been working with initrd secrets, so I am interested in the subject.

[RaitoBezarius]

@infinisil @amjoseph-nixpkgs I addressed the concern over the RFC being not interesting enough for this repo. I will ask you that we will at least mandate in this RFC the move to another repository.

@grahamc @cole-h Do you have any proposal for moving the RFC in another repository? I think either nixpkgs or nix-community or nixos would be the right place to do it. I also addressed various feedback on the initrd secrets and transition period.

@sdht0 I addressed your comments.

@tejing1 I addressed your comments.

I added @JulienMalka as a shepherd leader (after a IRL discussion). I am still waiting feedback from the other potential nominated shepherds.

[tejing1]

Much clearer regarding the initrd secrets. One thing could maybe do with a bit more clarification, though: Are the key strings of the initrd secrets hashmap meaningful to the bootloader backend's responsibilities at all? My inference from reading it is that they aren't, but it's a little vague.

[lheckemann]

@govanify @06kellyjac @hesiod @amjoseph-nixpkgs Could you confirm whether you accept your nominations as shepherds for this RFC?

[06kellyjac]

Yes I accept

[GovanifY]

I accept, sorry for the delay have been busier than anticipated!

[hesiod]

Thanks for the nomination, I accept.

[RaitoBezarius]

I updated everyone status on the specification. Please take the time to review the RFC text again, I will soon re-address the new comments. (I prefer to do it in batch.)

Once we are done with that, maybe, we can setup a quick call @JulienMalka to sync everyone on that RFC and see how we want to proceed.

I am also planning to do a prototype implementation before the RFC FCP as always.

[Luflosi]

When using flakes, one can add { system.configurationRevision = self.rev or "dirty"; } to their configuration. This allows figuring out which git commit of the flake was used to build the system by executing nixos-version --configuration-revision or nixos-rebuild list-generations. Can we add a field for the content of system.configurationRevision? Bootloaders could then also show this information in a useful way. Currently there is no easy way for a bootloader to show this information as the only place where it is currently stored is by being hardcoded into the nixos-version shell script.

Maybe this is already covered by somehow adding the information to the "label"?

[RaitoBezarius]

When using flakes, one can add { system.configurationRevision = self.rev or "dirty"; } to their configuration. This allows figuring out which git commit of the flake was used to build the system by executing nixos-version --configuration-revision or nixos-rebuild list-generations. Can we add a field for the content of system.configurationRevision? Bootloaders could then also show this information in a useful way. Currently there is no easy way for a bootloader to show this information as the only place where it is currently stored is by being hardcoded into the nixos-version shell script.

Maybe this is already covered by somehow adding the information to the "label"?

This is already covered by the label information IMHO, bootloaders decide what is the title and what information they need to encode for the label. Adding configuration revision seems out of scope just for cosmetic purpose.

[JulienMalka]

@RaitoBezarius @GovanifY @06kellyjac @hesiod @lheckemann let's set up a first sync call either the week of the 14th of the week of the 21st. Can you please fill this when2meet so that we can find an appropriate schedule for this? I think a 45mn call should be more than enough.

[JulienMalka]

@06kellyjac @hesiod I invited you both to a matrix room so that we can synchronously converge towards dates for our meeting, could you join please ?

[JulienMalka]

On February 22nd we had a RFC meeting. Participants: @RaitoBezarius and myself.

Overall, we agreed that there do not seem to be anything controversial preventing this RFC to go ahead. Most still open point are not big blockers and are handled asynchronously. The topic of providing structured information instead/alongside the label (see here) seem to be worthy of a synchronous discussion between stakeholders. We suggest scheduling a meeting to find a satisfactory way forward and this matrix channel can serve to organize such a meeting. (Ping @nikstur, @samueldr, @boomshroom, @RaitoBezarius)

During the meeting, we also decided that ideally, some implementation work should be done before the RFC get merged, mainly updating https://github.com/DeterminateSystems/bootspec and the NixOS module.

Migrations steps for the backends is also a topic that was discussed and that should be addressed in the RFC text, but it was discussed that to ensure backward and forward compatibility, migration between bootspec n-1 and n should be performed as follows:

1. bootloader backends should always be able to synthetize bootspec n from either nothing or bootspec n-1, to cover cases where a new backens sees an old generation, using https://github.com/DeterminateSystems/bootspec ;
2. the module system should emit bootspec n-1 and bootspec n, to cover for unmigrated backends, or old versions of backend seeing new generations.

[samueldr]

(I'm not a shepherd... Though I can be pinged/asked questions about boot given my domain knowledge.)

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-03-05/40851/1

[adamcstephens]

During the meeting, we also decided that ideally, some implementation work should be done before the RFC get merged, mainly updating https://github.com/DeterminateSystems/bootspec and the NixOS module.

Should this repository be moved under the NixOS organization?

[jonringer]

Hey shepherds, even if implementation is still ongoing, you can move this into FCP. The implementation doesn't need to be completed before this RFC has been merged.

Seems like there's agreement on the direction of this RFC; unless there are still outstanding concerns, the RFCSC recommendation would be to move to FCP.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-03-19/41829/1

[RaitoBezarius]

The FCP cannot happen before a first implementation is done in the ecosystem, IMHO.

Le mar. 19 mars 2024, 09:02, nixos-discourse @.***> a écrit :

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-03-19/41829/1

— Reply to this email directly, view it on GitHub https://github.com/NixOS/rfcs/pull/165#issuecomment-2007574071, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACMZRFWDPFP7PE3G7WGV4TYZBORRAVCNFSM6AAAAAA6ZOZO42VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBXGU3TIMBXGE . You are receiving this because you were mentioned.Message ID: @.***>

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-04-02/42643/1

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-04-16/43512/1