elemental
elemental copied to clipboard
rancher
Support for secure boot
I believe we are missing the shim image in our base image. This is something we need to support secure boot, also some adjustments on grub2 installation for secure boot might be required.
Just did a quick run of a teal build of system/cos including shim and mokutil packages and I can verify this is not enough to boot with secure boot enabled grub2-install does not do any magic if shim is installed in the system. Hence up to my understanding secure boot requires changes in elemental-cli to perform a proper bootloader installation including the shim image.
openSUSE and SLE provide the shim-install utility to install the shim.efi image and whatever is needed to load the self signed grub image. However shim-install appears to be a SUSE thing (it doesn't exist on Fedora) and it performs the grub2-install call in it. In elemental-cli we already run a grub2-install command that is tied to our custom use case, hence I wouldn't be surprised if the shim-install scripted procedure does not perfectly suit our use case. At glance, it looks like we might need to code our own secure boot install steps and being distro agnostic here is not obvious.
How much of a lift would this be if we target only Elemental Teal for secure boot? Then we can work on a vender agnostic version shortly after
ok, modifying a biot the grub-efi package I was able to boot on secureboot.
The trick is to have the shim as the boot entry. That chains into the grub.efi for the livecd ISO. And no matter what, if secureboot is disabled afterwards, the shim still works no problem.
The only problem is on how to add the boot entry, not sure who does that part....if we have to do it manually or uefi shell search for all paths or what. I believe in suse is done by the package for example, In my local pc I got an entry called opensuse-secureboot that is linked to the shim.efi.
The SUSE way of doing it is by calling the /usr/sbin/shim-install helper script which is part of the shim package. However I doubt this is an option for our use case, as this helper script is also calling grub2-install behind the scenes... In short, on a regular SUSE box you can call shim-install and this does everything from setting setting shim.efi and grub2 installation. I believe we just need to properly include shim.efi.
yeah shim install uses ~~mokutils~~ efibootmgr to remove all other entries and set its own entry. I guess we could do something like that if we are on EFI, just search for our own entry and add it if missing on install. That will allow us to boot from the installed system.
Now, Im not sure how the CD/DVD boots on EFI+secureboot...
Now, Im not sure how the CD/DVD boots on EFI+secureboot...
With some xorriso black magic... just run man xorriso, grab a cup of tea or coffee and start reading ;)
A hacky way of setting the shim entry using the shim-install script is by mocking grub2-install binary with /usr/bin/true and then call shim-install it will do everything without (re)installing grub2.
ok, I can see some issues here.
When we do a grub2-install we use the --removable flag, which is not really meant for a disk so the grub is set under /EFI/BOOT/BOOTX64.EFI which is the default name if there is nothing else.
This results into an efi system booting in there IF there is nothing else in the system. Basically its used for rescuing systems according to the docs. This also results into grub2-install not adding any entries into the bootloader!
We should probably install without that flag and set --bootloader-id=elemental so first, it sets the grub.efi into the proper dir EFI/elemental/grubx64.efi and then it adds the entry into the efibootmgr AND sets it to boot from there:
grub2-install: info: copying `/run/cos/state/boot/grub2/x86_64-efi/core.efi' -> `/run/cos/efi/EFI/elemental/grubx64.efi'.
grub2-install: info: Registering with EFI: distributor = `elemental', path = `\EFI\elemental\grubx64.efi', ESP at hostdisk//dev/vda,gpt1.
grub2-install: info: executing efibootmgr --version </dev/null >/dev/null.
grub2-install: info: executing modprobe -q efivars.
grub2-install: info: executing efibootmgr -c -d /dev/vda -p 1 -w -L elemental -l \EFI\elemental\grubx64.efi.
BootCurrent: 0001
Timeout: 3 seconds
BootOrder: 0008,0001,0002,0003,0000,0004,0005,0006,0007
Boot0000* UiApp
Boot0001* UEFI QEMU DVD-ROM QM00001
Boot0002* UEFI Misc Device
Boot0003* EFI Internal Shell
Boot0004* UEFI PXEv4 (MAC:5254008D938E)
Boot0005* UEFI PXEv6 (MAC:5254008D938E)
Boot0006* UEFI HTTPv4 (MAC:5254008D938E)
Boot0007* UEFI HTTPv6 (MAC:5254008D938E)
Boot0008* elemental
Installation finished. No error reported.
Then we should also add the same stuff as there is in the shim-install. Copy shim into the elemental dir, add bootloader entry (probably elemental-secureboot?)
That should lead to having proper support from efi. And down the line we may even support secureboot properly. But at least that would let us boot on unsecured platforms properly on UEFI.
So we require:
- sle-micro to have the shim package in the base
- elemental-cli to support installing the shim, disabling --removable, adding the entry elemental-secureboot to efi
- elemental-toolkit to ship shim in the live/grub2-efi packages
There is also some extra stuff in here.
I was able to install the shim with the proper bootloader-id with changes on elemental-cli. This results in an entry in the bootloader with the shim which properly boots our grub.efi on non-secureboot systems.
For us to be able to do secureboot we would need:
- Create a standalone grub.efi with all drivers included (insmod is forbidden in secureboot) using grub2-mkstandalone
- Sign the grub.efi with a valid signature
- Enroll the signature in the secureboot database
As we use the squash4 and loopback drivers for grub, we cant use the already-signed grub in the system unfortunately :/
BTW, this works for our ISO because we use the default grub, which is signed by suse. We do not modify it at all, so at least we cna run our iso in a secureboot environment LOL
Maybe we should check if we can have a fat-grub in the base image somehow that is signed, although I suspect that comes from the base SLE system, which makes it a bit more complicated?
This was a very interesting read http://www.rodsbooks.com/efi-bootloaders/index.html
also we probably need to modify install/update to add the grub.cfg into the ESP partition. Otherwise I have no idea how this thing searches for it? Seems like the default files from toolkit used to drop a grub.cfg that loads the config from the recovery partition??? https://github.com/rancher/elemental-toolkit/blob/main/packages/grub2/config/grub_efi.cfg.tmpl
But now it has nothing. Its just the grub.efi in there. I mean, it seems to work, maybe its due to the grub-install creation? This is something to check once we have a signed grub with loopback+squash4 modules bundled
Status: @kkaempf is looking to see what package we need to send a request to include 2 modules as bundled in grub, so we can have a suse signed grub in mainline that supports those modules(loopback, squash4) as they are critical to our boot (we mount loopback images to boot from)
Once that is in the next steps are:
- make elemental-cli copy the signed system grub to the ESP partition instead of generating one unsigned
- make elemental-cli copy the signed shim into the ESP partition
- make elemental-cli generate an entry in bootmager pointing to the shim
- check the grub.cfg status with the signed grub (picks it up automatically? doesnt? Do we need an extra grub.cfg in the ESP partition to chainload the active/passive/state grub?
OBS has it ready: https://build.opensuse.org/package/show/isv:Rancher:Elemental/grub2
tested with obs package and enrolling the generated cer (is on /usr/share/efi/x86_64/grub.cer along the grub.efi file) and it booted correctly under secureboot!
Some small changes needed on elemental-cli for the new shim -> grub -> chainload grub.cf from state. https://github.com/rancher/elemental-cli/pull/317
Update from grub2 maintainer:
It would be fine to add squash4 and loopback to signed image of 15SP4 since there's no tpm support in it. But I'd like to escalate that loopback mount from grub may not guarantee to work with tpm measure boot, given the potentially huge squashfs image has to be squeezed into memory entirely in one chunk to measure its hash into tpm before it can be used. In other words. grub used by ALP and openSUSE which has measure boot support may run into out of memory error if the squash rootfs is too large.
Besides more modules means to open more attack surface, so we have to be careful and doing that only when necessary.
unfortunately our whole boot process and A/B system relays on that loopback support to mount the active/passive images.
@davidcassany do you think we could do a different approach to that?
Im pretty ignorant on how the tpm measure boot works and if it applies to our A/B system fully but maybe we could do some kind of workaround for that, like having a middle loopback image that then mounts the final image so we keep the initial loopback mount small enough?
I dunno, arent most isos nowadays also using the squash stuff for the whole iso image that is mounted? How do they get away with that?
Is the squashfs size actually a concern for us ?
The current rootfs.squashfs is approx. 500MB and we haven't even started to optimize for size yet.
unfortunately our whole boot process and A/B system relays on that loopback support to mount the active/passive images.
@davidcassany do you think we could do a different approach to that?
Well, different approaches are possible, however I doubt we can keep it simple and backward compatible
- use A/B partitioning instead of A/B filesystem images. Impacts updates, installation and bootloader setup. Moreover a partition table is not the flexible and I am not sure about forcing the adoption of flexible approaches such as LVM. To my eyes it looks quite over complicated.
- use A/B btrfs subvolumes instead of filesystem images: while technically speaking this is not hard to achieve is likely to require a non backward compatible bootloader setup refactor. That would leave btrfs as the only supported file system for secure boot.
- extract kernel and initrd from the filesystem image: we could not longer tread the OS as a single image and has impact on install, upgrade and bootloader setup procedures. Also raises some questions about how to ensure the kernel and initrd match the image they pretend to boot.
I like the idea of the btrfs support and the use of subvolumes, but if something like that is implemented only to provide secure boot feels like an overkill. Also I dislike the message of secure boot is only supported for brtfs... I'd say we need loopback.
Regarding the image size issue with tpm measure boot. I'd say this is not a problem as long as there is no support for it in 15SP4, hence we are not using it at all. The concern on image sizes is valid since we are setting a loop device of the filesystem image, commonly in ext2, the default size is currently set to 3GB (I bet 2GB would be good enough for now too, also it would be nice to adapt the size of the image to the size of the root-tree, is a long standing nice to have feature).So I agree with grub2 maintainer that loopback and tpm might not work well together at grub level with out current setup. I don't think they are about to interact in any way though.
Isn't TPM at grub level used to decrypt disks? (I am far from familiar with TPM) I doubt this is something we can support now, feels like we might miss many pieces for a full disk encryption.
in any case secure boot adn measured boot are two different things. Secure boot only checks the signatures while measured boot checks the signatures and the hashes of the objects to load. Not only in measured boot we would encounter an issue with memory but also we would need to add our active/passive/recovery hashes to the TPM module on each update...doesnt seem to easily done. I would expect the measured boot to run up until grub+kernel is run only but maybe it also applies to devices mounted under grub, before the kernel is loaded :/
ugh, no it records everything what a pita:
https://www.gnu.org/software/grub/manual/grub/html_node/Measured-Boot.html
https://www.gnu.org/software/grub/manual/grub/html_node/Measured-Boot.html
This page talks about module loading - how's that possible with secure boot ?
gnu.org/software/grub/manual/grub/html_node/Measured-Boot.html
This page talks about module loading - how's that possible with secure boot ?
modules are also signed :+1: that is the reason we cannot use secureboot now, because if the modules were signed we would probably be able to load them
EDIT: NOT CONFIRMED; this is just a supposition based on several secureboot pages in which you are able to load signed modules
So we don't need a changed grub2 binary but signed squash4 and loopback modules would be sufficient ?!
So we don't need a changed
grub2binary but signedsquash4andloopbackmodules would be sufficient ?!
sorry, I made a mix of stuff in there.
the modules that are talked in that page are probably referring to the KERNEL modules, which need to be signed.
The modules we need to use are the grub modules, which cannot be signed individually, only signed as part of "bundling" them with the grub.efi as part of the grub build. Im not sure if we need to insmod them in the grub.cfg of they are already available out of the box to use them. There it comes my confusion when answering to this, sorry about that.
NOTICE: if we want to support compressed squash recovery, we need to add modules. for xz is xzio.mod, and Im guessing for gzip it would be gzip.mod.
We should be able to see what compression modules are shipped by default with the signed grub and mention in the docs that if you want a compressed squash recovery (Which IMO is useless, we dont get that much space gained, even with aggressive compression, but it slows down everything else like building and probably booting) we would only support the compression algs already part of the grub bundle.
(Which IMO is useless, we dont get that much space gained, even with aggressive compression, but it slows down everything else like building and probably booting)
👆🏻 this is the important part for documentation :wink:
gzio seems to be bundled so it should be possible to support squshfs compressed with gz under secure boot.
but IMHO we should discourage users from building compressed recoveries. Or at leat move form using the default xz compression udner elemental-cli to a gzip compression so we dont need any extra modules.