tracker
tracker copied to clipboard
turnkeylinux
Di-live support for UEFI install
Now that we've rebased our live ISO functionality on default Debian tools (i.e. live-tools & live-boot) and di-live is using default Debian d-i source code, suport for UEFI systems shouldn't be too hard...
I just realised that this is a duplicate of #932, but I'm going to close that in favour of this one.
I briefly had a look at this and I think it should be pretty easy to support UEFI (we'll need to include the d-i source for partman-efi and also include grub-efi package. What I think what might be a little tricky though, is support for both legacy BIOS and UEFI!
Having said that, it seems like it should be possible. Here are a few links which might help:
- https://wiki.debian.org/UEFI
- https://superuser.com/questions/801515/is-a-hybrid-linux-usb-stick-for-uefi-legacy-bios-possible
- https://linuxconfig.org/legacy-bios-uefi-and-secureboot-ready-ubuntu-live-image-customization
- https://www.0xf8.org/2020/03/recreating-isos-that-boot-from-both-dvd-and-mass-storage-such-as-usb-sticks-and-in-both-legacy-bios-and-uefi-environments/
- https://wiki.syslinux.org/wiki/index.php?title=Isohybrid
I had hoped to implement this in v17.0, but we've run out of time...
Having said that, I have got the ISO UEFI boot part sorted - see the cdroots uefi-boot branch and this fab PR (which undid the working UEFI boot supporting ISO creation process).
I did do some work on di-live too, but that was never completed. The outstanding work can be found in this PR.
In order to get bitkey working again I also needed uefi live "cd" boot.
Checked out uefi-boot branch in tkldev cdroots and patched product.mk to use run-genisoimage-uefi instead of run-genisoimage. This worked fine and got an iso image created which I dd'ed onto a usb stick.
Tested those on a few machines and can report some positive feeback: Working on an older Intel Desktop with UEFI boot mode enabled as well as disabled. Worked on a rather new Dell XPS 13 with UEFI boot enabled, and also worked on an old Dell laptop without UEFI capabilities.
Where it didn't work, but where I wanted it most, was a Surface Pro 3 Laptop which only has UEFI boot mode (thanks alot microsoft ;-) ). The usb stick just did not boot. So I thought I'd do the most basic and try to produce a simple UEFI only usb stick for testing on the Surface. So I created a gpt partition table, a fat32 partition and tagged this as EFI system partition in fdisk. Then, I copied boot, isolinux (most probably not necessary) and live directory (where kernel and rootfs is) from the generated hybrid iso onto the stick.
First idea was to use the bootx64.efi from here and copy that into the default efi binary path /efi/boot/bootx64.efi onto the stick. That resulted in some unknown filesystem errors. I guess the gpt partition and fat modules are not in there. So my next steps in this direction would probably be to regenerate them via grub-mkrescue as adviced in the uefi-boot branch readme and add some more grub modules.
However, second idea was to directly use grub-mkimage as advised here: https://askubuntu.com/questions/395879/how-to-create-uefi-only-bootable-usb-live-media/395880#395880
I just changed the prefix path (where grubs additional files will reside on the target partition) to match the iso:
sudo grub-mkimage -o bootx64.efi -p /boot/grub -O x86_64-efi fat iso9660 part_gpt part_msdos normal boot linux configfile loopback chain efifwsetup efi_gop efi_uga ls search search_label search_fs_uuid search_fs_file gfxterm gfxterm_background gfxterm_menu test all_video loadenv exfat ext2 ntfs btrfs hfsplus udf
This probably has more modules inside than needed, but that does not matter. I got it to boot into the correct turnkey grub menu screen, so I was sure that /boot/grub/grub.cfg was loaded. But when selecting the "boot from live image / tryout" option I got this:
Loading Linux ...
error: symbol `grub_efi_secure_boot' not found.
looking at the grub.cfg I might try to add insmod part_gpt, but I am not sure if this is the problem. I also tried to look for the string "grub_efi_secure_boot" inside the grub source code tree but it wasn't found, which leaves me a bit puzzled. If anyone knows the root cause of this error message, I would be very interested...
so far, I will try to update here if I got more results.
Nice work @derMart! :smile: It sounds like you've had some good progress.
One thing, is it possible to disable secure boot on the Surface? (It wouldn't surprise me if it wasn't but perhaps double check? If your UEFI stuff is working elsewhere, then my guess is that it's choking on secure boot).
And if it is choking on secure boot, then you'll need to boot from a boot loader that is signed by a key that is acceptable to the UEFI - we don't have such a key so can't help you there. I suggest starting with a Debian Live ISO and see how that goes. If that boots ok, then perhaps you can hack on that to chainload BitKey?
thanks alot @JedMeister, yes progress, even more so :-). I forgot to mention that it is possible to disable secure boot on the surface and I already did so. However, grub efi file was created on the host os (not the vm tkldev is running), maybe that host efi binary accepts signed modules only and when doing the actual insmod it was trying to load a .mod file generated from the vm, which is non signed, just a suspicion.
Regarding the initial issue with the tkldev product.iso and the surface: I suspect, that it just doesn't like the isohybrid-gpt-basdat approach and as such didn't eat the efi.img. I looked what options grub-mkrescue is actually invoking on xorriso and basically copied their approach with this:
xorriso -as mkisofs \
-V $ISOLABEL \
-r $O/cdroot \
-b boot/grub/i386-pc/eltorito.img \
-no-emul-boot -boot-load-size 4 \
-boot-info-table \
--grub2-boot-info \
--grub2-mbr /usr/lib/grub/i386-pc/boot_hybrid.img \
--efi-boot efi.img \
-efi-boot-part --efi-boot-image \
--protective-msdos-label \
-o $O/product.iso \
--sort-weight 0 / \
--sort-weight 1 /boot
This works to boot live mode on all my available machines :-) Old BIOS only laptop, UEFI only surface, UEFI/BIOS desktop and UEFI/BIOS laptop.
The nice thing about it is, that it only uses grub and does not rely on isolinux which reduces complexity. I am not sure though, if it will work when burned on an actual DVD, but also I don't see why not. At least, I tried it as removeable optical medium inside Virtualbox and that worked fine too. grub-mkrescue is actually adding more options to get hfs plus, but I skipped it as I did not get it to produce an image. No idea about how hfs works or if this grub image will work on old macs, but I would suspect that newer macs are actual UEFI compatible (?)
If you think this grub only approach is worth it, I am happy to generate a pull request to replace run-genisoimage-uefi.
On the other side, or additional, it might be worth to somehow allow in the build process of a product to choose from different run-genisoimage methods if there won't be a one solution fits all approach. But I am not that deep into the build process to make a suggestion how to implement such switch best.
Wow. You've got a long way and I'm pretty impressed to be honest.
I forgot to mention that it is possible to disable secure boot on the surface and I already did so. However, grub efi file was created on the host os (not the vm tkldev is running), maybe that host efi binary accepts signed modules only and when doing the actual insmod it was trying to load a .mod file generated from the vm, which is non signed, just a suspicion.
Hmm, ok. Well if it's not secure boot that's the issue, this may be irrelevant, but FWIW it appears that there is a Microsoft signed shim available in Debian and signed grub and kernel packages too! These are the packages that might be worth having a look at? I doubt it's exhaustive, but could be worth further investigation? That is if you wanted to see if you can get secure boot working:
- grub-efi-amd64-signed-template
- grub-efi-amd64-signed-template
- shim-signed
- shim-helpers-amd64-signed-template
- shim-helpers-amd64-signed-template
I know nothing about them, but I (loosely) monitor the Debian bugtracker and I saw this bug this morning. The bug itself is not directly relevant (it's related to ARM64 specifically) but it bought the existence of those packages to my attention.
I looked what options grub-mkrescue is actually invoking on xorriso and basically copied their approach [...] This works to boot live mode on all my available machines :-) Old BIOS only laptop, UEFI only surface, UEFI/BIOS desktop and UEFI/BIOS laptop.
That's awesome!
If you think this grub only approach is worth it, I am happy to generate a pull request to replace run-genisoimage-uefi.
So long as it continues to work on vanilla VMs (i.e. without requiring anything special) and legacy hardware then I'm more than happy with that. I was under the impression that we'd essentially need to support 2 methods - one for UEFI and one for legacy/vanilla VM stuff. So that's a very pleasant surprise! And I'd welcome a pull request. We won't be implementing any of this until we rebase on Debian 12/Bookworm (currently Debian 'testing' - i.e. once it becomes 'stable'). But just open a PR against master for now and I'll create new dev branches and edit your PR to be against the dev branch (once I create it).
On the other side, or additional, it might be worth to somehow allow in the build process of a product to choose from different run-genisoimage methods if there won't be a one solution fits all approach. But I am not that deep into the build process to make a suggestion how to implement such switch best.
We already have so many different targets/images I'd rather not duplicate everything again if it can be avoided. So if I understand correctly and you're saying that the run-genisoimage-uefi method works for both VMs/legacy hardware and UEFI, then I'm super happy with that.
@JedMeister 🙏 Thank you very much! That secure boot stuff is interesting, but as everything is booting currently, I am ok with this for the moment and will turn my attention to the main dish of getting bitkey working ;-)
We already have so many different targets/images I'd rather not duplicate everything again if it can be avoided. So if I understand correctly and you're saying that the run-genisoimage-uefi method works for both VMs/legacy hardware and UEFI, then I'm super happy with that.
Yep, according to my tests it does. You are also right in that it still are two methods. One is that there is a GPT partition table with an EFI system partition, and the other is that on the respective offset in the ISO an MBR is sitting for old BIOS machines. Difference is that grub is now handling both approaches :-)
Here is the PR: https://github.com/turnkeylinux/fab/pull/18
I have tested this successfully, except for cdroot-dynamic target, but I guess that should work, I just removed the isolinux directory on the target cdroot. Which is what also can be done in the cdroots repo, but I am leaving that for you.
best derMart
Hi @derMart - Thanks very much. I've just posted on your PR and hopefully we'll get some other eyes over it shortly, although it looks good to me! :)
Good luck with the BitKey work. Please feel free to ask if you hit any more issues and definitely let me know if you can get it working! I'd love to publish an updated build and am more than happy to give you your due credit.
@truefriend-cz - that's correct.
The work that @derMart has done so far means that the ISO can boot on a UEFI system, but the install still won't work (because the installed system can't boot with UEFI - only legacy BIOS).
As such, we haven't merged this code yet.
We are still working on it, and it may make it into v18.x at some point (once we work it out) but for now we don't support UEFI hardware unfortunately. That sucks for sure, although for context, 99% of TurnKey deployments are to VM/container/Cloud, so whilst I get that's cold comfort for you if you want to install on recent hardware, it's not very common, so we can't really justify the time and effort to focus on it.
oh... god I just lost almost 4h trying to install Turnkey-Core on bare metal Dell Wyse 5070 :(. The only thing I can do now is to follow this thread.
Hi @krzakx - sorry to hear of your frustration and wasted time :cry:
FWIW, whilst our custom minimalist install made a ton of sense when we first developed it (~12 years ago), that is no longer the case so much. Now that the "official" Debian Live system exists and has matured (it didn't when we first migrated to/rebased on Debian) and the default Debian installer now supports installing from a live system (again it didn't back then - one of many reasons why we developed our custom installer). After evaluating our options (and considering the work required to make our custom installer di-live work and then maintain it into the future) we're considering deprecating it and just using the default Debian installer. Whilst that will resolve this particular issue (and a few others) there is still quite a bit of work involved, so I can't give, even a rough ETA.
In the meantime, I highly recommend installing a hypervisor, such as Proxmox to your bare metal (which should go smoothly; Proxmox has great hardware support - as that is it intended installation target). Then you can install TurnKey on top. We partner with Proxmox to provide TurnKey LXC builds via the Proxmox UI (web UI or CLI) or you can install TurnKey from ISO to a "proper" VM (via QEMU/KVM in Proxmox). Given equivalent resources, LXC has virtually no overhead (compared to bare metal). Modern QEMU/KVM has approximately 5-10% overhead, so even if running as a "proper" VM the resource usage difference should not be casually noticeable.
Obviously there are plenty of other hypervisor options too (e.g. VMware vSphere) although IMO Proxmox is the way to go (but I'm clearly biased... :grin: )
Wanted to use this on a small box PC, but it's a fairly recent device, so I stumbled over this too. As it seems there isn't even a proper workaround (the only one I found is installing debian first and then installing TKL over it somehow, but that post was a few years old), which is tough...
