Add Raspberry Pi image generation support

[ArrayBolt3]

Adds Raspberry Pi installation and image generation support. Cross-builds are supported (you can create a working Raspberry Pi image on an amd64 machine).

Some notes:

• raspi-firmware generates the cmdline.txt for the kernel in a highly non-ideal way, using a device path rather than a UUID for identifying the root directory. This has to be manually patched to get the image to boot at all.
• If your host system is a Pi using one of the raspi.debian.net images, you probably want to pass --defaultinterfaces to grml-debootstrap, since the /etc/network/interfaces file used in the raspi.debian.net images is just a pointer to /etc/network/interfaces.d, which grml-debootstrap doesn't copy over. Using --defaultinterfaces will prevent this from being a problem, not using it will mean you'll have no network when you boot the system.
• The image DOES NOT automatically resize itself on first boot. The RPi image generation is more like VM image generation, the image is fixed-size and you have to resize it afterwards. If you don't want that, you can install directly to a USB drive or SD card.
• This uses the current way of doing things as implemented in the normal Debian Raspberry Pi images, where the kernel is booted directly by the firmware bootloader. There are lots of other ways to boot a Pi (involving u-boot and/or edkII), which might be more suitable for some use cases. However, as grml-debootstrap is intended to install pure Debian, I tried to stick with a pure Debian approach as much as possible. I've also done research showing that a direct-kernel-boot install like this can be converted to a U-Boot + GRUB install without too much effort. (See https://www.kicksecure.com/wiki/Dev/boot#Booting_Debian_Trixie_with_GRUB_.2B_u-boot_on_Raspberry_Pi_4)

Fixes https://github.com/grml/grml-debootstrap/issues/114

[ArrayBolt3]

Found a bug in grml-debootstrap that existed before this PR - installing a VM image to a device such as /dev/sda was impossible because grml-debootstrap was trying to format devices named /dev/mapper/sda1p1 (which is wrong for multiple reasons). This was because it always assumed that device names were formatted as {disk_driver}{disk_number}p{partition_number}, when some devices are formatted as {disk_driver}{disk_letter}{partition_number}. Fixed in an added commit (https://github.com/grml/grml-debootstrap/pull/335/commits/752a6114ee5f82a34177b2bde7622115b70213ac), that one should be able to be cherry-picked without merging this entirely.

[zeha]

Found a bug in grml-debootstrap that existed before this PR - installing a VM image to a device such as /dev/sda was impossible because grml-debootstrap was trying to format devices named /dev/mapper/sda1p1 (which is wrong for multiple reasons). This was because it always assumed that device names were formatted as {disk_driver}{disk_number}p{partition_number}, when some devices are formatted as {disk_driver}{disk_letter}{partition_number}. Fixed in an added commit (752a611), that one should be able to be cherry-picked without merging this entirely.

AFAIK, VM images were never supposed to be written to devices by grml-debootstrap. @mika ?

[ArrayBolt3]

@zeha By installing a VM image directly to a device, I mean using a command line such as --target /dev/sda --vm, not using --vmfile. I probably just used the wrong terminology.

[mika]

@zeha By installing a VM image directly to a device, I mean using a command line such as --target /dev/sda --vm, not using --vmfile. I probably just used the wrong terminology.

Hm, what's the use case for --target /dev/sda --vm exactly, as in: why install an VM into a block device at all and not into a plain file only? (Not claiming that there's no use case for it, but to clarify actual needs and how far we should go with supporting that :))

[ArrayBolt3]

Off the top of my head, I know people oftentimes store VMs in LVM volumes rather than in files. I don't know for sure why, but I suspect performance plays a role.

For the Raspberry Pi case specifically, being able to install directly to a physical device would be useful to prevent someone from having to generate an image, flash the image, and then resize the image to occupy the entire disk. They can just install straight to the device and use it.

On Tue, Apr 8, 2025 at 1:39 AM Michael Prokop @.***> wrote:

@zeha https://github.com/zeha By installing a VM image directly to a device, I mean using a command line such as --target /dev/sda --vm, not using --vmfile. I probably just used the wrong terminology.

Hm, what's the use case for --target /dev/sda --vm exactly, as in: why install an VM into a block device at all and not into a plain file only? (Not claiming that there's no use case for it, but to clarify actual needs and how far we should go with supporting that :))

— Reply to this email directly, view it on GitHub https://github.com/grml/grml-debootstrap/pull/335#issuecomment-2785391393, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZAFFERDXAATHKG3WH2BST32YNVLBAVCNFSM6AAAAAB2P3MTM6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOOBVGM4TCMZZGM . You are receiving this because you authored the thread.Message ID: @.***> [image: mika]mika left a comment (grml/grml-debootstrap#335) https://github.com/grml/grml-debootstrap/pull/335#issuecomment-2785391393

@zeha https://github.com/zeha By installing a VM image directly to a device, I mean using a command line such as --target /dev/sda --vm, not using --vmfile. I probably just used the wrong terminology.

Hm, what's the use case for --target /dev/sda --vm exactly, as in: why install an VM into a block device at all and not into a plain file only? (Not claiming that there's no use case for it, but to clarify actual needs and how far we should go with supporting that :))

— Reply to this email directly, view it on GitHub https://github.com/grml/grml-debootstrap/pull/335#issuecomment-2785391393, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZAFFERDXAATHKG3WH2BST32YNVLBAVCNFSM6AAAAAB2P3MTM6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOOBVGM4TCMZZGM . You are receiving this because you authored the thread.Message ID: @.***>

[mika]

Off the top of my head, I know people oftentimes store VMs in LVM volumes rather than in files. I don't know for sure why, but I suspect performance plays a role. For the Raspberry Pi case specifically, being able to install directly to a physical device would be useful to prevent someone from having to generate an image, flash the image, and then resize the image to occupy the entire disk. They can just install straight to the device and use it.

Ah I see, thx4info!

[zeha]

JFTR, this is even documented in man grml-debootstrap:

  --vm
      Set up a Virtual Machine on an existing block device, which will be partitioned. This
      allows deployment of a Virtual Machine. The options needs to be combined with the
      --target option. This option automatically enables the --defaultinterfaces option. 
      Usage example: --vm --target /dev/mapper/your-vm-disk

(Note the example.)

[ArrayBolt3]

Ran thorough tests to ensure things worked as expected. "Working" is defined as "the build doesn't error out, the produced image is bootable without having to take any special steps, the root user account can be logged into, the contents of /boot are appropriate for the installation being tested, the partition layout is appropriate for the installation being tested, and apt update successfully downloads an updated package database, ensuring network connectivity works. It may be required to modify /etc/network/interfaces to get the interface name right, this is expected."

• Builder hardware: x86_64 VM
  • [x] Can create a working x86_64 BIOS VM image? sudo grml-debootstrap --release bookworm --password 'x' --force --defaultinterfaces --arch amd64 --vmfile --target ./x86_64-bios-bookworm.img
  • [x] Can create a working x86_64 UEFI VM image? sudo grml-debootstrap --release bookworm --password 'x' --force --defaultinterfaces --arch amd64 --vmfile --vmefi --target ./x86_64-efi-bookworm.img
  • [x] Can create a working x86_64 BIOS VM direct install? sudo grml-debootstrap --release bookworm --password 'x' --force --defaultinterfaces --arch amd64 --vm --target /dev/vdb
  • [x] Can create a working x86_64 UEFI VM direct install? sudo grml-debootstrap --release bookworm --password 'x' --force --defaultinterfaces --arch amd64 --vm --vmefi --target /dev/vdb
  • [x] Can create a working arm64 UEFI VM image? sudo grml-debootstrap --release bookworm --password 'x' --force --defaultinterfaces --arch arm64 --vmfile --vmefi --target ./arm64-efi-bookworm.img
  • [x] Can create a working arm64 UEFI VM direct install? sudo grml-debootstrap --release bookworm --password 'x' --force --defaultinterfaces --arch arm64 --vm --vmefi --target /dev/vdb
  • [x] Can create a working RPi image? sudo grml-debootstrap --release bookworm --password 'x' --force --defaultinterfaces --rpifile --non-free --target ./arm64-rpi-bookworm.img
  • [x] Can create a working RPi direct install? sudo grml-debootstrap --release bookworm --password 'x' --force --defaultinterfaces --rpi --non-free --target /dev/vdb
• Builder hardware: arm64 Raspberry Pi 4
  • [x] Can create a working arm64 UEFI VM image? grml-debootstrap --release bookworm --password 'x' --force --defaultinterfaces --arch arm64 --vmfile --vmefi --target ./arm64-efi-bookworm.img
  • [x] Can create a working arm64 UEFI VM direct install? grml-debootstrap --release bookworm --password 'x' --force --defaultinterfaces --arch arm64 --vm --vmefi --target /dev/sda
  • [x] Can create a working RPi image? grml-debootstrap --release bookworm --password 'x' --force --defaultinterfaces --rpifile --non-free --target ./arm64-rpi-bookworm.img
  • [x] Can create a working RPi direct install? grml-debootstrap --release bookworm --password 'x' --force --defaultinterfaces --rpi --non-free --target /dev/sda

I did fix a couple of bugs in the original PR as I went, but both fixes required small, non-invasive changes.

[ArrayBolt3]

Kicksecure isn't really thrilled with the boot process that raspi-firmware uses by default, due to its effects on things like booting into older kernels, and customizing kernel parameters at boot time. If possible, we'd like to be using U-Boot and grub-efi-arm64 instead. I filed some feature requests upstream (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1102607, https://salsa.debian.org/raspi-team/image-specs/-/issues/78) to see if this can be changed. If things do change in that area and GRUB + U-Boot booting for the Pi is officially supported in Debian, could grml-debootstrap use that boot flow instead, or at least offer it as an option? (Obviously we're more than happy to send the patch for this, I just want to make sure this will be acceptable in the future so we can do project planning.)

[ArrayBolt3]

@mika, @zeha Gentle ping, anything else that needs done to this?

[mika]

@mika, @zeha Gentle ping, anything else that needs done to this?

I'm personally not really a huge fan of Raspberry devices, but I assume there are users out there who might benefit from it. So IMO it's also about how intrusive this change would become, and how we can ensure that things keep working. Looking at e.g. https://github.com/ptrsr/pi-ci, is this something we could somehow also provide through our GitHub CI? :thinking:

[ArrayBolt3]

I'm not sure, I'm not exactly handy with Docker, and based on the documentation it looks like pi-ci is both an emulator and image generator, there doesn't appear to be a documented easy way to boot a custom image with it. Newer versions of QEMU have support for emulating the Raspberry Pi 4B, which might be able to be used somehow, but even then you have to specify the kernel and initramfs to boot directly (QEMU doesn't emulate the boot firmware's behavior of reading config.txt and figuring out what to boot from that). I guess it would be possible to kinda sorta emulate the boot firmware by mounting the build image, finding config.txt, parsing out the kernel and initramfs lines, copying those out of the image, then unmounting the image and booting the kernel + initramfs + image with QEMU. I'd be willing to give that a shot, though be warned that I'm not particularly experienced with CI systemd either :P

[adrelanos]

It's published on dockerhub so the CI image does not have to be built from source code at least. Not sure that is still relevant...

Newer versions of QEMU have support for emulating the Raspberry Pi 4B, which might be able to be used somehow, but even then you have to specify the kernel and initramfs to boot directly (QEMU doesn't emulate the boot firmware's behavior of reading config.txt and figuring out what to boot from that).

So such as test would not directly test if either config.txt based kernel direct boot is functional. Let alone if GRUB based booting is functional.

I guess it would be possible to kinda sorta emulate the boot firmware by mounting the build image, finding config.txt, parsing out the kernel and initramfs lines, copying those out of the image, then unmounting the image and booting the kernel + initramfs + image with QEMU.

That sounds rather complex and may not be very useful as it would not test on the CI if the image is actually bootable. It would test kernel + initramfs and anything after that. Not sure how useful that would be?

Is CI testing mandatory?

[mika]

That sounds rather complex and may not be very useful as it would not test on the CI if the image is actually bootable. It would test kernel + initramfs and anything after that. Not sure how useful that would be?

Is CI testing mandatory?

What we like and appreciate about CI testing is, that we catch any possible side-effects/breakages with any changes within g-d very early. It definitely improves our options to redesign/rewrite things, as well as accept contributions. It's not a 100% "must have" for everything all the time, but it's something we definitely prefer having. :)

[zeha]

That sounds rather complex and may not be very useful as it would not test on the CI if the image is actually bootable. It would test kernel + initramfs and anything after that. Not sure how useful that would be? Is CI testing mandatory?

What we like and appreciate about CI testing is, that we catch any possible side-effects/breakages with any changes within g-d very early. It definitely improves our options to redesign/rewrite things, as well as accept contributions. It's not a 100% "must have" for everything all the time, but it's something we definitely prefer having. :)

To add my thought: everything that's not CI tested will break. When we'll do a refactor, nobody will test stuff that's not automatically tested, and it will just be broken. My personal view is that it's close to mandatory.

[mika]

That sounds rather complex and may not be very useful as it would not test on the CI if the image is actually bootable. It would test kernel + initramfs and anything after that. Not sure how useful that would be? Is CI testing mandatory?

What we like and appreciate about CI testing is, that we catch any possible side-effects/breakages with any changes within g-d very early. It definitely improves our options to redesign/rewrite things, as well as accept contributions. It's not a 100% "must have" for everything all the time, but it's something we definitely prefer having. :)

To add my thought: everything that's not CI tested will break. When we'll do a refactor, nobody will test stuff that's not automatically tested, and it will just be broken. My personal view is that it's close to mandatory.

nod && ACK!

[ArrayBolt3]

kk, then I guess I'll be adding CI to this :+1:

[ArrayBolt3]

Unfortunately, it looks like CI is going to end up being very slow if we want to use QEMU's RPi4 emulation. QEMU is unable to use KVM when emulating a Raspberry Pi 4B because the CPU in the RPi supports EL3, which can be emulated but not virtualized using KVM and QEMU. https://gitlab.com/qemu-project/qemu/-/issues/1073 has some more info on this. It is possible to boot a generated Raspberry Pi image using a more VM-friendly machine type, but it doesn't result in as accurate of Raspberry Pi emulation, which might be undesirable.

Should I use the faster but less accurate mode, or the slower, more accurate one?

[ArrayBolt3]

I'll go ahead and implement this using the faster but less accurate virtualization. It's all that's necessary to make sure the image was built properly, the accurate mode is probably only necessary for making sure the kernel or firmware itself is functional on the official hardware itself (which isn't what we need to test, that should be being tested by other projects).

[ArrayBolt3]

Raspberry PI image CI testing added. https://github.com/ArrayBolt3/grml-debootstrap/actions/runs/14696085489 will show the results once the tests are done running. (I got them to pass fully in a separate branch first.)

[ArrayBolt3]

@mika, @zeha, gentle ping, this should be ready for review. RPi image CI tests have been added and are passing.

[ArrayBolt3]

While working on the docs, I'm realizing the --vm mode (which the --rpi mode is modeled after) automatically enables --defaultinterfaces. I think this makes a lot of sense for Raspberry Pi images. I'll try to sneak that in there before this is merged.

[ArrayBolt3]

Tested the new default interfaces commit on my RPi4, works like a charm, don't need to do anything special for the network to work out of the box. CI passes and documentation is updated.

[ArrayBolt3]

Hmm, that's a good point. I didn't realize environment variables were a legal way to pass in parameters. That may require some rework.

On Sun, May 4, 2025 at 11:54 AM Michael Prokop @.***> wrote:

@mika commented on this pull request.

---

In tests/docker-build-vm.sh:

grml-debootstrap
--debug
--force \

• --vmfile \
• --vmsize 3G \

• ${extra_buildopts} \
• --imagesize 3G \

I see. I'm not yet sure about this change, while I agree that consistency is important and if we keep it backwards compatible at least for now that should be fine. @zeha what's your take on the --vmfile vs --imagesize situation? 🤔

(BTW, AFAICS if someone uses VMSIZE=4G as environment variable on the command line, then this is no longer accepted due to the [ -n "$VMSIZE" ] || VMSIZE="2G" removal, nor?)

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[ArrayBolt3]

Reworked, tested locally, VMSIZE and IMAGESIZE environment variables are being respected as appropriate now.

[mika]

Thanks, LGTM. I think we should be good to apply this, though I'm afraid it's nothing we can get into Debian/trixie at this stage. To not block ourselves for further changes/fixes of grml-debootstrap that are supposed to go into Debian/trixie, we need to postpone this merge until the trixie freeze is gone.

Assigning this to our 2025/Q3 project/milestone accordingly, any further testing/feedback appreciated in the meanwhile! :)

[ArrayBolt3]

Just rebased this on the current git master. Looks like CI passes: https://github.com/ArrayBolt3/grml-debootstrap/actions/runs/20687641909 Currently testing a cross-build from amd64, will also test "build on RPi, booted on RPi" and see what happens.

[mika]

Just rebased this on the current git master. Looks like CI passes: https://github.com/ArrayBolt3/grml-debootstrap/actions/runs/20687641909 Currently testing a cross-build from amd64, will also test "build on RPi, booted on RPi" and see what happens.

Thanks, please ping us once you think it's ready for review @ArrayBolt3

Our package-build / build debian action currently always seems to fail with:

RequestError [HttpError]: Resource not accessible by integration

Is that a GH issue or do we need to adjust/fix something, @zeha? :thinking:

[ArrayBolt3]

@mika, @zeha I believe this is ready for review. Both an amd64 cross-build (installing directly to an SD card) and an arm64 native build (installing directly to a USB drive) worked. Both installations were bootable on a Raspberry Pi 4B (and the USB native installation was done from within the amd64 cross-build installation).