Issue During Reboot After Running `sdm-cryptconfig`
Ran sdm-cryptconfig from a running freshly booted RPi 500+, received all the expected output and rebooted.
Made it to part in the process marked by the red line in this picture stolen from the examples:
After entering the passphrase, I get the below x2, and then a new passphrase prompt:
Cannot initialize device-mapper. Is dm_mod kernel module loaded?`
Cannot use device cryptroot, name is invalid or still in use.
The first line I recall the material pre-reboot said was ok, the second line is interesting. I have let it hang out for a while to try to evaluate whether it's still encrypting (it's a 256GB NVMe drive). Been probably about 45 min at the time of writing this.
I still suspect that it's encrypting (due to personal experience plus the warning line that encryption can take a while), but posting this just on the off chance that there's anything to do other than give it time (or if there's a suspicion that something else could have gone awry). Wondering if there's a way to present any kind of progress indicator (my guess is no during this level of boot) or maybe the script can query whether the device is ready prior to requesting the passphrase (unless the way to do that is to try to unlock it haha).
Unfortunately the dd command in busybox (used in initramfs) doesn't appear to support status=progress, so no easy solution.
You didn't mention which OS you're using. This process works well on Bookworm, but I just ran a test on Trixie only to discover that something is not working well, so I'm now off working on that.
Thanks for the prod! ๐
Good callout, it's Trixie haha (of course).
Happy to blitz it and try again if you've got anything in the works to test.
Well, there are changes in cryptsetup for Trixie, need to sort out the required new magic incantations.
I'll post here when there's something for you to try.
It turns out that this is an old bug that has resurfaced.
This issue first showed up 5 months ago (or so), and after a debug session with the cryptsetup guys and a filed bug against the RasPiOS kernel it was fixed.
For Bookworm.
Unfortunately, seems to be slightly broken on Trixie in that only the 4Kb page kernel works. 16Kb kernel elicits what you've already seen.
In the meantime, if you want to give it a try again on your 500+, re-install Trixie on it and make the following changes to /boot/firmware/config.txt:
- Add
kernel=kernel8.img - Verify that
auto_initramfs=1is enabled
Reboot, then start the encryption process. This worked fine for me.
Good thing it's a new 500+ with not much on it! ๐
I'll follow-up on the kernel issue.
Adding the kernel=kernel8.img line and keeping auto_initramfs=1 and rebooting after, then starting sdm didn't work. I'll try your original edit of the comment in the morning.
RaspberryPi issue: https://github.com/raspberrypi/trixie-feedback/issues/5
Adding the
kernel=kernel8.imgline and keepingauto_initramfs=1and rebooting after, then starting sdm didn't work. I'll try your original edit of the comment in the morning.
Huh. I'll retest later as well.
Later edit: Confirmed. I used kernel=kernel8.img and auto_initramfs=1, rebooted, ran sdm-cryptconfig and the system is now happily running with an encrypted rootfs with 4Kb pagesize.
If you are unable to get it to work, please provide a very detailed sequence of steps you did so I can try to repro it. Starting with as close to a fresh install as possible is always a good thing.
Previous attempts were definitely very close to a fresh install, only things I think I ran post first-login were apt update, apt full-upgrade, rpi-update (realizing as I type this that maybe this is the culprit), rpi-eeprom-update, rpi-keyboard-fw-update, and some uninstalls (chromium and the default installed IDE applications).
Using the Raspberry Pi Imager 1.9.6 (I know I know, it's not sdm, but I just discovered your tool ๐ ), I write the image (not lite, not full recommended apps, the one with just the desktop environment and some bare bones apps). I apply a hostname, locale settings, username/password, and leave everything else off (telemetry, SSH, etc.).
This time, I shall just fresh install, sudo apt update && sudo apt full-upgrade, modify the /boot/firmware/config.txt file, sudo systemctl reboot, sudo sdm-cryptconfig, then sudo systemctl reboot again.
Running this, straight from your instructions to grab sdm-cryptconfig:
# sdm-cryptconfig is required for rootfs encryption
sudo curl --fail --silent --show-error -L https://github.com/gitbls/sdm/raw/master/sdm-cryptconfig -o /usr/local/bin/sdm-cryptconfig
sudo chmod 755 /usr/local/bin/sdm-cryptconfig
sudo sdm-cryptconfig
Post-reboot, in the initramfs prompt:
sdmcryptfs /dev/nvme0n1 /dev/sda
Answer YES, enter passphrase, chill for a bit, enter passphrase again, and now it appears to be progressing as expected.
Exiting the initramfs shell boots normally, performs the cleanup, auto-reboots, and prompts for the passphrase as expected. Performed another reboot for kicks, behaves as expected.
I have typed all this up, so I may as well submit it for posterity. I guess that I didn't realize that rpi-update was only for bleeding edge shenanigans, and I suspect that it was the culprit here after you provided the manual fix for the regression.
Since you're here and I have your attention, side question, since some Googling seems to indicate that one could configure a FIDO2 hardware key (I'm thinking specifically of Yubikeys) to decrypt the volume instead of a passphrase, maybe you could add that to your tool ๐ ? I have several Yubikeys, including at least one Bio, and it would be pretty sleek to be able to configure Pi images with your tool that prep not only all the fun stuff you already do, but with LUKS2 encryption unlocked by one of those devices.
Thank you for the help (including revealing my PEBCAK issue haha) and for the tool that I'm going to start using ๐ !
Glad you found the culprit!
Some comments on this, from top to bottom of your post:
- You should NEVER do an rpi-update unless you know what you're doing, and have determined that you need it to correct an issue. Likely the culprit, as you noted. I've been using Pis for 10 years or so (7 with high usage), and I've used rpi-update twice.
- Likewise, you shouldn't need to do an rpi-eeprom-update. When there are updates via apt they'll be installed.
- I have no clue what rpi-keyboard-fw-update is, but it's probably in the same bucket as the above. Unless you KNOW you need it, just. don't.
- WRT to using rpi-imager vs sdm to burn disks:
- rpi-imager is fine, but it does some stuff you may not want (especially with Trixie, using cloud-init)
-
sdm does some things that you may want, and miss out on when you don't use it to burn disks
- Expands the rootfs on the disk after burning, unless you tell it not to. One less thing to do when the system boots
- Optionally Run plugins you specify after the disk is burned, enabling you to build unique, customized system disks from a single IMG. I use this so that my servers are built from a common IMG but have different services configured on them
- Optionally create additional partitions on the disk after burning
- Optionally, sdm can burn the disk such that the rootfs is btrfs or LVM.
- Lastly, sdm's configuration capabilities for SSH, multiple users, and networks are much more flexible than those of rpi-imager and can be applied during customization or burning the disk.
WRT unlocking the encrypted rootfs wth a yubikey, it's been on my todo list for quite some time, and I have a yubikey sitting on my desk. The last time I looked at it I concluded that it was a bit more complex than I was willing to tackle at the time due to other stuff needing attention. The difficult part, of course, is that the initramfs environment is not the most hospitable, so debugging new bits can be frustrating and challenging at times.
I'm planning to have another look at this sometime soon-ish, but in the meantime you can
- Use a key stored on a USB disk to unlock
- Provide the unlock passphrase via SSH
I tried the encryption process this morning with a Pi5 and 16Kb pages (ie NO kernel=kernel8.img) and it worked fine.
I don't know what changed, but since all of my attempts started with a freshly-customized Trixie IMG I can only assume that some package got recently updated and magically fixed it.
Wonderful! Thank you for the update!
I am planning to give your tool a go to customize an image and then I'll redeploy and see how it goes with regards to encryption.
Selfishly I hope that you find the time to work through the FIDO2 key fiddly bits!
Wonderful! Thank you for the update!
I am planning to give your tool a go to customize an image and then I'll redeploy and see how it goes with regards to encryption.
Selfishly I hope that you find the time to work through the FIDO2 key fiddly bits!
Great! Looking forward to hearing how it goes.
And, yes, I will have a look at yubikey again, now that Trixie is working, some rough edges have been smoothed over, and perhaps most importantly, the Big Dark is about to descend on us PNWers. ๐
I spent some time looking at yubikey support. It turns out that it's pretty easy to manually add it to an already-encrypted rootfs.
I'd really appreciate it if you would give this a try and LMK your feedback. NB I only tested this on Trixie. Also, this disables sdm-cryptconfig's keyfile (on a USB stick).
Steps to add a yubikey to an already-encrypted disk:
- Install the yubikey software:
sudo apt install yubikey-luks yubikey-personalization
- Plug in the yubikey
- Set up the yubikey
sudo ykpersonalize -2 -ochal-resp -ochal-hmac -ohmac-lt64 -oserial-api-visible
- Look at the luks dump to ensure that slot 1 is free. It should be. (/dev/sdX2 is the encrypted rootfs partition)
sudo cryptsetup luksDump /dev/sdX2
- Add the yubikey to the free slot (1 in this case)
sudo yubikey-luks-enroll -d /dev/sdX2 -s 1
The output will look something like this:
setting disk to /dev/mmcblk0p2.
setting slot to 1.
This script will utilize slot 1 on drive /dev/mmcblk0p2. If this is not what you intended, exit now!
Adding yubikey to initrd
Please insert a yubikey and press enter.
Please enter the yubikey challenge password. This is the password that will only work while your yubikey is installed in your computer:
Please enter the yubikey challenge password again:
Please provide an existing passphrase. This is NOT the passphrase you just entered, this is the passphrase that you currently use to unlock your LUKS encrypted drive:
WARNING: The --key-slot parameter is used for new keyslot number.
- Configure crypttab with the yubikey keyscript
sudo sed -i "s#discard#discard,keyscript=/usr/share/yubikey-luks/ykluks-keyscript#" /etc/crypttab
- Update initramfs with the changes we just made
sudo update-initramfs -u
That's it. Reboot and the console should print:
Please insert yubikey and press enter or enter a valid passphrase
The prompt doesn't seem quite right to me, because you need to enter the passphrase you entered above in the yubikey-luks-enroll command after you insert the yubikey BEFORE you press enter, but I suppose most of us won't read the text after the first time or two ๐คฃ
Also, I have not yet been able to get the unlock to work by pressing the yubikey. Needs some in-depth investigation.
If you don't insert the yubikey you can use the unlock passphrase you provided during the encryption process. Unlock with this passphrase also works via SSH if you configured that with sdm-cryptconfig
Lastly, This website was extremely helpful: https://www.endpointdev.com/blog/2022/03/disk-decryption-yubikey/ although I was unable to get the suggested change to /etc/ykluks.cfg to work in the brief testing I did. More required, I guess.
At this point I'm still thinking about whether it's worth doing any automation of this or whether an addendum in sdm's encryption guide is sufficient. I can certainly clean up the unlock code, which seems a bit janky, although I may get a fresh appreciation for it after messing around with it. ๐ฒ
Appreciate your thoughts...
Hmm...more futzing. If you give it a try, set up a scratch system, IE something you don't care about. There's something not quite right, although it does work if I type in the unlock password for the yubi.
I haven't done anything meaningful yet with the 500+ so I will tool around with that haha.
I skimmed through your previous post just now and I'll give it a more thorough go tomorrow and give it a shot!
Quick questions:
- Which YubiKey did you use? I have...too many. I have some 4's, 5's, Bio's, and some Security Keys (the Security keys ONLY do FIDO2/WebAuthn and FIDO U2F).
- Did you just use the YubiKey to provide ("type") a passphrase? Or is it using FIDO2? (I read through this and some other things and it seems like folks use a systemd utility: https://forum.endeavouros.com/t/using-fido2-keys-to-unlock-luks-on-endeavouros/51111 )
I haven't done anything meaningful yet with the 500+ so I will tool around with that haha.
I skimmed through your previous post just now and I'll give it a more thorough go tomorrow and give it a shot!
Quick questions:
- Which YubiKey did you use? I have...too many. I have some 4's, 5's, Bio's, and some Security Keys (the Security keys ONLY do FIDO2/WebAuthn and FIDO U2F).
- Did you just use the YubiKey to provide ("type") a passphrase? Or is it using FIDO2? (I read through this and some other things and it seems like folks use a systemd utility: https://forum.endeavouros.com/t/using-fido2-keys-to-unlock-luks-on-endeavouros/51111 )
I was using a yk5NFC.
This is the yk providing the passphrase, not fido2. I figured this would be easy to do, but I just (after a bit too much time screwing around with it) discovered that there's a bug in the unlock script that prevents password-less use. Been there since 2023: https://forums.linuxmint.com/viewtopic.php?t=393163
Once I get this working I'll have a look at fido.
I now have the challenge-response (aka "type") working. I'm building a plugin that one would run on the booted system AFTER sdm-cryptconfig has completed to configure this.
I also spent some time attempting to get fido2 working but so far no success. If you can find a web page like the one you found for endeavouros, BUT for Debian, that would be VERY helpful.
Once the challenge-response plugin is working will you be able to test it?
Yes! I meant to test a couple days ago but alas, work got in the way haha.
Tonight or tomorrow are blissfully free so I want to allocate time to this. Part of me was annoyed at doing it on the 500+ popping the case every time to remove the NVMe drive to test, but I had a moment of clarity and realized that I have an SD card and the device will boot from those too ๐. So I can leave the SSD intact and just go crazy nuking and re-imaging the SD card to test.
Link vomit from trying to find useful things. In order of my perception of usefulness:
Links that actually mention Debian:
- https://www.guyrutenberg.com/2022/02/17/unlock-luks-volume-with-a-yubikey/
- https://www.matuck.com/tech/2023/09/03/Debian-12-with-LUKS-and-Fido2.html
- https://support.nitrokey.com/t/unlocking-luks-volume-with-nitrokey-3-on-debian-12-2/5553
Links that don't mention Debian but DO mention systemd-cryptenroll:
- https://kudelskisecurity.com/research/luks-disk-encryption-with-fido2
- https://blog.fraggod.net/2023/01/26/more-fido2-hardware-authkey-uses-on-a-linux-machine-and-their-quirks.html
- https://0pointer.net/blog/unlocking-luks2-volumes-with-tpm2-fido2-pkcs11-security-hardware-on-systemd-248.html
Other things I found enlightening:
- https://manpages.debian.org/unstable/systemd-cryptsetup/systemd-cryptenroll.1.en.html
- https://security.stackexchange.com/questions/280791/fido2-yubikey-systemd-cryptenroll-options-differences-of-fido2-with-client
- This one is interesting to me because I have a YubiKey Bio, so it seems like I'd be able to set
fido2-with-user-verification=trueandfido2-with-client-pin=falsemaybe and only need my fingerprint (which obviously requires my physical presence) to unlock. Though, I would probably still require the PIN, but I like that it's an option.
- This one is interesting to me because I have a YubiKey Bio, so it seems like I'd be able to set
Yes! I meant to test a couple days ago but alas, work got in the way haha.
Tonight or tomorrow are blissfully free so I want to allocate time to this. Part of me was annoyed at doing it on the 500+ popping the case every time to remove the NVMe drive to test, but I had a moment of clarity and realized that I have an SD card and the device will boot from those too ๐. So I can leave the SSD intact and just go crazy nuking and re-imaging the SD card to test.
OK! I'll have a draft version before I hang it up for the night. Docs will be...terse
OK, first version for testing. Drop the unzipped plugin into /usr/local/sdm/plugins
The yubi plugin configures an already-encrypted rootfs to be unlocked with a Yubikey in addition to the rootfs passphrase. This plugin MUST be run live on a booted system, as it requires access to the Yubikey hardware.
Arguments (*=required)
- *
lukskeyโ key used when the luks partition was encrypted - *
partitionโ encrypted partition (e.g., /dev/sdX2) - *
ykphraseโ yubikey passphrase -
noautounlockโ Require a typed passphrase (ykphrase) to unlock. Default is auto-unlock -
initializeโ Initialize the Yubikey. Default is to not -
luksslotโ Luks keyslot to use. Default is 2 (which is free on a newly-encrypted rootfs)
Steps
- Start with a freshly burned, freshly encrypted disk. Use the cryptroot plugin or sdm-cryptconfig
- After the system is fully booted we can configure the yubikey
- The plugin requires youbikey access. If the yubikey is not found it will prompt for it
sdm --runonly plugins --oklive --plugin yubi:"luksey=mykey|partition=/dev/sda2|ykphrase=myyubipassphrase"
After this completes, reboot. If the yubikey is plugged in, the system should perform a complete boot. If the yubikey is not present you can type in the luks rootfs encryption password or insert the yubikey.
A couple things I notice from https://developers.yubico.com/yubikey-personalization/ :
We have officially announced the End of Life of the YubiKey Personalization package on February 19, 2025, in line with Yubicoโs End-of-Life policy. the YubiKey Personalization package will reach its End of Life on February 19, 2026.
Warning: By using this tool you will destroy the AES key in your YubiKey. This prevents it from being useful against Yubicoโs validation server. It is possible to upload a new AES key to Yubico, using a random YubiKey prefix, to restore it. But it is not possible to get back your old yubikey prefix if you decide to re-program your YubiKey.
That second one seems dire, but they don't really explain why I should care if the key is "...useful against Yubico's validation server." Maybe it's for this?
- https://www.yubico.com/genuine/
I think the first warning is probably more relevant to building out this functionality.
A couple things I notice from https://developers.yubico.com/yubikey-personalization/ :
We have officially announced the End of Life of the YubiKey Personalization package on February 19, 2025, in line with Yubicoโs End-of-Life policy. the YubiKey Personalization package will reach its End of Life on February 19, 2026.
Warning: By using this tool you will destroy the AES key in your YubiKey. This prevents it from being useful against Yubicoโs validation server. It is possible to upload a new AES key to Yubico, using a random YubiKey prefix, to restore it. But it is not possible to get back your old yubikey prefix if you decide to re-program your YubiKey.
That second one seems dire, but they don't really explain why I should care if the key is "...useful against Yubico's validation server." Maybe it's for this?
- https://www.yubico.com/genuine/
I think the first warning is probably more relevant to building out this functionality.
I apparently missed that particular warning. The yubi documentation seems fairly contorted for a yubi newbie like me ;)
The packages are in Debian 13, so I'm not going to worry about it for now. As far as the AES keys, a point worth noting in the plugin documentation.
After more digging around, it seems like the functionality is present in the ykman (Yubikey Manager CLI) tool:
- https://developers.yubico.com/yubikey-manager/
- https://docs.yubico.com/software/yubikey/tools/ykman/OTP_Commands.html
That tool also appears to be present in the Debian 13 repos.
As an update to the testing conversation, I had to learn your tool a bit but I just successfully (I think) used a more bare bones version of your ezsdm script (removed the apps plugin lines to save time, set username/pass to "test", and added the cryptroot plugin) to customize an image and I just burned it to SD.
The lukskey argument in your new plugin, to make sure that I understand, that's the original passphrase that was used to encrypt the disk in the first run, yes?
After more digging around, it seems like the functionality is present in the
ykman(Yubikey Manager CLI) tool:
- https://developers.yubico.com/yubikey-manager/
- https://docs.yubico.com/software/yubikey/tools/ykman/OTP_Commands.html
That tool also appears to be present in the Debian 13 repos.
I'm going to be away from my pi playground for a bit, but will have a look at it when I get back. Certainly willing to switch if it's not a heavy lift.
As an update to the testing conversation, I had to learn your tool a bit but I just successfully (I think) used a more bare bones version of your
ezsdmscript (removed the apps plugin lines to save time, set username/pass to "test", and added thecryptrootplugin) to customize an image and I just burned it to SD.The
lukskeyargument in your new plugin, to make sure that I understand, that's the original passphrase that was used to encrypt the disk in the first run, yes?
Correct. It's the passphrase you typed to sdmcryptfs during the initial encryption.
So, I ran the below to customize the image:
#!/bin/bash
function errexit() {
echo -e "$1"
exit 1
}
[ $EUID -eq 0 ] && sudo="" || sudo="sudo"
img="$1"
[ "$img" == "" ] && errexit "? No IMG specified"
[ "$(type -t sdm)" == "" ] && errexit "? sdm is not installed"
#[ "$sudo" != "" ] && assets="." || assets="/etc/sdm/local-assets"
assets="."
rm -f $assets/my.plugins.1
[ -f $assets/my.plugins ] && mv $assets/my.plugins $assets/my.plugins.1
(cat <<EOF
# Plugin List generated $(date +"%Y-%m-%d %H:%M:%S")
EOF
) | bash -c "cat >|$assets/my.plugins"
(cat <<'EOF'
user:deluser=pi
user:adduser=test|password=test
disables:piwiz|cloudinit
L10n:host
cryptroot
EOF
) | bash -c "cat >>$assets/my.plugins"
$sudo sdm --customize --plugin @$assets/my.plugins --extend --xmb 2048 --restart --regen-ssh-host-keys $img
Then ran:
sudo sdm --burn /dev/mmcblk0 --hostname testpi1 --expand-root 2025-10-01-raspios-trixie-arm64.img
Everything seemed to go well, it did everything I was expecting when it rebooted, but after the encryption process, after typing exit in the initramfs cli, it booted to the login screen without requiring the passphrase, and then after about a minute rebooted and hung at the Raspberry Pi boot sequence (it made it past the red and white screen where it selected the boot device, but stayed on the black screen with a central window with the RPi logo).
So, I'm going to recustomize and try again in case there was a one-off reason that I didn't make it to the finished product.
Sounds like you're trying it with a Desktop version? Might I suggest that you try it with a Lite version so that you get a good feel for how things are supposed to work?
In the encryption guide, in the "initramfs" section, the block that starts with "sdmcryptfs will then:" describes in detail the steps that will occur.
Since there are multiple reboots and complex things happening, it would be helpful if you described any issues you run into using these steps as a framework (and now I see that having them numbered might be a good thing for exactly this reason!)
That said, after sdmcryptfs finishes, the disk has already been unlocked, and when you type exit to the initramfs prompt it simply continues the boot already in progress (it's not a reboot), so there's no need to unlock the rootfs again.
Yea I tried it with the "basic" desktop version, not the "recommended" that comes with all the extra apps. Same thing happened when I reburned it, so now I will take your advice and swap to the lite .img and go again!
First system boot: At the end, the sdm-firstboot service runs, which completes sdm system configuration and reboots
An sdm FirstBoot script, /etc/sdm/0piboot/099-enable-auto-encrypt.sh runs and configures the service sdm-auto-encrypt to run on reboot
Second boot: At the end of system startup, the sdm-auto-encrypt service runs which reconfigures the system for encryption and reboots
The sdm-auto-encrypt service runs sdm-cryptconfig to configure the system for encryption and configures the sdm-cryptfs-cleanup service
Third boot: System startup drops into initramfs to perform the encryption process
In initramfs you use the sdmcryptfs command to encrypt rootfs
Upon initramfs exit, system continues booting
At end of system startup the sdm-cryptfs-cleanup service tidies sdm-related services on the running system, removes sdmcryptfs-related bits from the initramfs, and then reboots
NOTE: You MUST wait for this stage to automatically reboot or the encryption will fail
It went through first, second, and third boot, reboots at the end, and then hangs on the loading screen after that. But we shall see what the lite version brings!
Alrighty. With a Lite version, stuff worked, all good.
- Booted
- Unlocked LUKS partition
- Boot continues
- Login
-
sudo apt updateandsudo apt full-upgradefor kicks - Installed sdm using the
curlcommand from the main page -
curl'ed the yubi.zip and extracted - Copied to
/usr/local/sdm/plugins/ - Plugged in YubiKey 5 NFC (USB A)
- Validated that it is accessible to the system generally speaking using
sudo ykman infoandsudo ykman list - Ran the provided command:
-
sudo sdm ---runonly plugins --oklive --plugin yubi:"lukskey=test|partition=/dev/mmcblk0p2|ykphrase=test"
-
- Exits with failure status, previous line is "Yubikey must be connected before running this plugin"
- I opened up the plugin file and I see only one line with that phrase, and it looks (to me) like it's checking to see if a
1is returned byykinfo -q -2 - Ran
ykinfo -q -2directly and I got a0, so that is my guess as to why it's choking there
Going to keep poking around to see if I can figure out anything else.
ok, that's interesting. What is the output from an lsusb command with the yubikey plugged in?