Hello, i have tried like 20 times to install pimox but the error is always the same:

Done. Loading new ceph-dkms-0.0.2 DKMS files... It is likely that 5.15.30-v8+ belongs to a chroot's host Building for 5.15.30-v8+ Building initial module for 5.15.30-v8+ Error! Bad return status for module build on kernel: 5.15.30-v8+ (aarch64) Consult /var/lib/dkms/ceph-dkms/0.0.2/build/make.log for more information. dpkg: error processing package ceph-dkms (--configure): installed ceph-dkms package post-installation script subprocess returned error exit status 10 Setting up pve-manager (7.1-10) ... Job for pvestatd.service failed because the control process exited with error code. See "systemctl status pvestatd.service" and "journalctl -xe" for details. dpkg: error processing package pve-manager (--configure): installed pve-manager package post-installation script subprocess returned error exit status 1 Setting up zfs-dkms (2.0.3-9) ... Removing old zfs-2.0.3 DKMS files...

---

Deleting module version: 2.0.3 completely from the DKMS tree.

Done. Loading new zfs-2.0.3 DKMS files... It is likely that 5.15.30-v8+ belongs to a chroot's host Building for 5.15.30-v8+ Building initial module for 5.15.30-v8+

configure: error: *** None of the expected "capability" interfaces were detected. *** This may be because your kernel version is newer than what is *** supported, or you are using a patched custom kernel with *** incompatible modifications. *** *** ZFS Version: zfs-2.0.3-9 *** Compatible Kernels: 3.10 - 5.10

Error! Bad return status for module build on kernel: 5.15.30-v8+ (aarch64) Consult /var/lib/dkms/zfs/2.0.3/build/make.log for more information. dpkg: error processing package zfs-dkms (--configure): installed zfs-dkms package post-installation script subprocess returned error exit status 10 dpkg: dependency problems prevent configuration of proxmox-ve: proxmox-ve depends on pve-manager; however: Package pve-manager is not configured yet. proxmox-ve depends on ceph-dkms; however: Package ceph-dkms is not configured yet.

dpkg: error processing package proxmox-ve (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of zfs-zed: zfs-zed depends on zfs-modules | zfs-dkms; however: Package zfs-modules is not installed. Package zfs-dkms which provides zfs-modules is not configured yet. Package zfs-dkms is not configured yet.

dpkg: error processing package zfs-zed (--configure): dependency problems - leaving unconfigured Processing triggers for initramfs-tools (0.140) ... Errors were encountered while processing: ceph-dkms pve-manager zfs-dkms proxmox-ve zfs-zed E: Sub-process /usr/bin/dpkg returned an error code (1) root@rasp1-prox:~#

I followed every single step in this guide (and i tried even other guides) but the installation in never "successful".

My hardware:

Raspberri Pi 4 model B 8GB RAM 32GB Samsung SD Card

Tried even the os image described in the installation script but nothing changed, the error is still the same. I really don't know what to do next or what to try :/

Have exactly the same errors and can not complete the installation on the Pi4.

None of the expected "capability" interfaces were detected. *** This may be because your kernel version is newer than what is *** supported, or you are using a patched custom kernel with *** incompatible modifications.

---

*** ZFS Version: zfs-2.0.3-9 *** Compatible Kernels: 3.10 - 5.10

Error! Bad return status for module build on kernel: 5.15.30-v8+ (aarch64)

Installed kernel is too new for installation.

Everything else is a consequential error, because the dependencies are missing. Here one should times nachbesser or use kernel 5.10!

Have exactly the same errors and can not complete the installation on the Pi4.

Hi, after trying like 25 times, i finally found out the problem (maybe, i think). You need to expand the root partition, after flashing the os ssh into the Pi, while you are in "pi" user, du "sudo raspi-config" > advanced > expand root partition, then reboot. After reboot, as user "pi" di "sudo apt update -y && sudo apt upgrade -y", after all of this, you can run the install script.

There are many errors but proxmox works (somehow), but...is really difficult to boot an effective vm inside it, is note realy stable. To upload the .iso file i needed to try 4 times, everytime failed. Anyway, i recommend you to erad this ENTIRE post BEFORE you try https://raspberrytips.com/proxmox-on-raspberry-pi/

This has less to do with the Expanded Filesytem under raspi-config.

The installation should already go through without errors, because after each apt update dmks ceph will try to build again. Remedy is simply to use a new kernel -> 5.10.95

https://github.com/raspberrypi/linux/tree/rpi-5.10.y https://www.raspberrypi.com/documentation/computers/linux_kernel.html

Build: sudo apt install git bc bison flex libssl-dev make git clone --depth=1 --branch rpi-5.10.y https://github.com/raspberrypi/linux cd linux KERNEL=kernel8 make bcm2711_defconfig -j4

Config: sudo apt install libncurses5-dev make menuconfig -j4

KSM Support active

Kernel 64Bit Build: make -j4 Image modules dtbs sudo make modules_install sudo cp arch/arm64/boot/dts/broadcom/.dtb /boot/ sudo cp arch/arm64/boot/dts/overlays/.dtb* /boot/overlays/ sudo cp arch/arm64/boot/dts/overlays/README /boot/overlays/ sudo cp arch/arm64/boot/Image /boot/$KERNEL.img

Reboot

Done ;-)

Well thanks! I tought it was this just because the panel won't load up before.

I also ran into problems with zfs-dkms when setting up Pimox7 using RPiOS64-IA-Install.sh on a freshly installed 2021-10-30-raspios-bullseye-arm64-lite, RPI 4B 8GB.

So I decided to manually setup Pimox7 essentially doing the same thing the script above does but skipping the installation of the package raspberrypi-kernel-headers and I got through the whole process withouth errors. Could also successfully install ceph and everything was looking real fine.

Wanted to give the install script another run, this time removing && apt install -y raspberrypi-kernel-headers from line 148. It worked perfectly!

Could not spend much time on real debugging but wanted to share my trial&error results here :)

Hello, somebody has been able to start a vm in a rpi4 ? I followed every guide but the vm just won't start up, it throws some error like:

BdsDxe: failed to load Boot0001 "UEFI QEMU QEMU CD-ROM " from PciRoot(0x0)/Pci(0x5,0x0)/Scsi(0x0,0x2): Not Found BdsDxe: failed to load Boot0003 "UEFI QEMU QEMU HARDDISK " from PciRoot(0x0)/Pci(0x5,0x0)/Scsi(0x0,0x0): Not Found

Start PXE over IPv4.

I'm trying to virtualize home assistant image right now

With regard to @topa-LE kernel build instructions...

sudo cp arch/arm64/boot/dts/broadcom/.dtb /boot/ sudo cp arch/arm64/boot/dts/overlays/.dtb* /boot/overlays/

This should be...

sudo cp arch/arm64/boot/dts/broadcom/*.dtb /boot/
sudo cp arch/arm64/boot/dts/overlays/*.dtbo /boot/overlays/

And then you still need to do... make headers_install

Also, before installing the proxmox-ve package, remove at least the raspberrypi-kernel-headers package (leaving only the headers from the just-compiled kernel) or dpkg will still try to build the modules for the packaged kernel as well.

Hello, somebody has been able to start a vm in a rpi4 ? I followed every guide but the vm just won't start up, it throws some error like:

BdsDxe: failed to load Boot0001 "UEFI QEMU QEMU CD-ROM " from PciRoot(0x0)/Pci(0x5,0x0)/Scsi(0x0,0x2): Not Found BdsDxe: failed to load Boot0003 "UEFI QEMU QEMU HARDDISK " from PciRoot(0x0)/Pci(0x5,0x0)/Scsi(0x0,0x0): Not Found

Start PXE over IPv4.

I'm trying to virtualize home assistant image right now

hi - ha works with generic aarch64 starting with 8+, see the other issue.

https://github.com/pimox/pimox7/issues/63

haos_generic-aarch64-8.0.dev20220325.img works fine. do not use the rp4 native images, you need a uefi arm64/aarch64 image/iso.

Instead of compiling the kernel from scratch I was able to install Pimox7 with the last 5.10 kernel from the official archive without issues. The downside of this is that you won't get any kernel updates anymore. Let's see when Proxmox and Pimox moves on to Kernel 5.15 and beyond 😄

wget http://archive.raspberrypi.org/debian/pool/main/r/raspberrypi-firmware/raspberrypi-kernel_1.20220120-1_arm64.deb
wget http://archive.raspberrypi.org/debian/pool/main/r/raspberrypi-firmware/raspberrypi-kernel-headers_1.20220120-1_arm64.deb

sudo dpkg -i raspberrypi-kernel_1.20220120-1_arm64.deb raspberrypi-kernel-headers_1.20220120-1_arm64.deb

sudo apt-mark hold raspberrypi-kernel raspberrypi-kernel-headers

sudo -s
curl https://raw.githubusercontent.com/pimox/pimox7/master/RPiOS64-IA-Install.sh > RPiOS64-IA-Install.sh
chmod +x RPiOS64-IA-Install.sh
./RPiOS64-IA-Install.sh

Thanks, @ToSMaverick!

To make things a little easier, you can start with 2022-01-28-raspios-bullseye-arm64-lite which includes the 20220120 kernel. You would still need to download and install the 20220120 version of the kernel-headers package.

Another small change that may not really make a difference: I noticed that the raspberrypi-bootloader package had the same datestamp version as kernel and kernel-headers, so I used apt-mark hold on that as well.

I had the same issue. It would be great to have an uninstaller script to revert all the changes. Are we supposed to reflash rpios or are the changes not greater than different a hostname, hosts file and an additional interfaces?

it says it can't find the kernel Building for 5.15.32-v8+ as supported, only supporting Compatible Kernels: 3.10 - 5.10.

According to https://github.com/openzfs/zfs/issues/12786, 5.15 should be supported though?

The great thing with ToSMaverick's header install is that it can fix your automated install script install after the fact.

it says it can't find the kernel Building for 5.15.32-v8+ as supported, only supporting Compatible Kernels: 3.10 - 5.10.

According to https://github.com/openzfs/zfs/issues/12786, 5.15 should be supported though?

I think I was able to fix it with a recommendation from another issue (like removing a line or something like that).

Even then, pixmox was not really runnable nor was it properly documented.

Switching to VMware ESXi was the best decision. Literally no errors, super easy and it actually works in the end... Follow a tutorial on youtube regarding datastores.

Another solution is to download the ceph-dkms repository and replace with the current kernel code. The procedure is the following:

1. First you need to clone RPi's Kernel source code.

git clone https://github.com/raspberrypi/linux.git --depth=1

2. Then copy all ceph files into a new default-src directory.

mkdir -p default-src/drivers/block
cp -r linux/drivers/block/{rbd.c,rbd_types.h} default-src/drivers/block/
grep -E "^#|^ccflags|rbd.o" linux/drivers/block/Makefile > default-src/drivers/block/Makefile
mkdir -p default-src/fs/ceph
cp linux/fs/ceph/* default-src/fs/ceph/
mkdir -p default-src/net/ceph
cp -r linux/net/ceph/* default-src/net/ceph/

3. Now you need to clone the original repository

git clone https://github.com/pimox/ceph-dkms.git

4. Then you need to replace the default-src directory with the one you created in the step before

rm -rf ceph-dkms/src/default-src
cp -r default-src ceph-dkms/src/default-src

5. To make sure the deb won't get (easily) replaced, we can increase its minor version. This step is optional and can cause issues with future updates, but I recommend it.

(
  cd ceph-dkms/
  sed -i "s/0\\.0\\.2/0.0.2-1/g" debian/changelog
  sed -i "s/0\\.0\\.2/0.0.2-1/g" debian/dkms
)

6. Now you need to regenerate the .deb

(cd ceph-dkms && make)

7. You can finally install it using dpkg. Note: if you didn't change the version before, you need to update the package version below.

(
  cd ceph-dkms
  sudo dpkg -i ceph-dkms_0.0.2-1_all.deb
)

For the zfs-dkms, you can just follow OpenZFS's instructions for installing on Debian Bullseye.

As of today, at least, this seems to be fully working. Just tested on both nodes on my cluster and they all seem pretty happy with the new packages:

$ apt list --installed | grep -E "(ceph|zfs)-dkms"

ceph-dkms/now 0.0.2-1 all [installed,local]
zfs-dkms/bullseye-backports,bullseye-backports,now 2.1.4-1~bpo11+1 all [installed,automatic]

Have fun!

Edit: typos.

Well, I just ordered another pi to get into the clustering fun, and will try out that new way, while this time also documenting everything step for step.

I'll write it down at Gist: luckydonald/pi_proxmox_homeassistant_docker.md, and I will keep it up to date there.

Hi all, I'm stumbling into an issue with the automated install. My rpi4 get stuck (blank screen, no other tty possible, network not available) on DEBIAN_FRONTEND=noninteractive apt install -y -o Dpkg::Options::="--force-confdef" proxmox-ve

I tried pinning the headers as mentioned in https://gist.github.com/luckydonald/1849291fb5e19c87df8c8a1618e29eaa#1-install-pimox-proxmox-on-the-raspberry-pi but it didn't help et all.

Also tried: https://github.com/pimox/pimox7/issues/66#issuecomment-1126826946 But didn't work either - install wasn't successful :/

I'm using an external ssd via usb boot with this pi. Could it be possible that my issue is related to this?

@deB4SH It shouldn't be an issue, as I'm using external SSDs to boot both my Pis as well. Are you sure you're using Raspbian OS 64bit (https://github.com/pimox/pimox7#install-from-scratch-rpios64bit-interactive-automatic-installer)?

You can also try to install manually as per the README - it isn't much more complicated.

@rgsilva yes the latest image flashed by the rpi-imager. also latest bios

@deB4SH that's unusual. Try installing over a local tty instead of SSH, so you can switch between consoles and check what is going on. It could be also power issues (dmesg should warn you about that, plus the power LED usually indicates issues as well), as I had some trouble with that in the past that caused exactly those issues: blank screen, no tty, no network.

@rgsilva do I have to do those steps before or after the installer?

Also the line

cp -r linux/drivers/block/{rbd.c,rbd_types.h} sdefault-srcrc/drivers/block/

should be (sdefault-srcrc)

cp -r linux/drivers/block/{rbd.c,rbd_types.h} default-src/drivers/block/

@luckydonald after. You can check if it's broken by checking the result of the installation to see if the other packages failed to install.

Ooops, sorry about the typo! I've fixed it!

To install pimox on top of 5.15 kernel, you need to install zfs 2.1 To achieve that, you need

• first to install linux-headers matching your kernel
• then to configure apt to use bullseye-backports repo as documented here https://openzfs.github.io/openzfs-docs/Getting%20Started/Debian/index.html#installation

apt install zfs-dkms/bullseye-backport

# apt install zfsutils-linux/bullseye-backport

apt install zfs-zed/bullseye-backport

Result can be checked

apt list | grep zfs | grep installed

libzfs4linux/bullseye-backports,now 2.1.5-1~bpo11+1 arm64 [installed,automatic] zfs-dkms/bullseye-backports,now 2.1.5-1~bpo11+1 all [installed] zfs-zed/bullseye-backports,now 2.1.5-1~bpo11+1 arm64 [installed] zfsutils-linux/bullseye-backports,now 2.1.5-1~bpo11+1 arm64 [installed]

Then there is another similar trick to solve with ceph-dkms dependecy. this package needs to be compiled as root on your system https://github.com/pimox/ceph-dkms

Pulling my hair out over this.

Confused as to when this should be done in relation to the actual PiMox install.

It seems I've tried every combination, and I'm failing miserably.

RPi 4, 8G, Raspberry Pi OS 64 Lite

Thanks mucho!

@GneeChee - install the Raspberry Pi OS, follow the steps in https://github.com/pimox/pimox7/issues/66#issuecomment-1099110466, then run this repository's installer script.

EDIT: Attached combined log files pimox-install-log.txt

@alanivey - thanks for the response ...

Freshly created/updated RPi 64 OS Lite -- this is what I got when I followed @ToSMaverick post:

No errors from his portion of the setup.

From the pimox install script -

Errors were encountered while processing: pve-manager proxmox-ve E: Sub-process /usr/bin/dpkg returned an error code (1)

After rebooting the pi - apt upgrade - I get this -

pimox@PiMox:~ $ sudo apt upgrade Reading package lists... Done Building dependency tree... Done Reading state information... Done Calculating upgrade... Done The following packages have been kept back: raspberrypi-kernel raspberrypi-kernel-headers 0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded. 2 not fully installed or removed. After this operation, 0 B of additional disk space will be used. Do you want to continue? [Y/n] y Setting up pve-manager (7.2-7) ... Setting up proxmox-ve (7.2-1) ... ln: failed to create symbolic link '/boot/pve/vmlinuz': Operation not permitted ln: failed to create symbolic link '/boot/pve/initrd.img': Operation not permitted pimox@PiMox:~ $

So - still not working, for me :(

You can try to follow my guide for that, if you like: https://gist.github.com/luckydonald/1849291fb5e19c87df8c8a1618e29eaa#file-1-pimox-md (permalink)

Sorry for the delay in responding, and thanks for the link.

I tried option A ... no bueno, batman :(

I also should mention that I tried the new install script that fixes the very issue that I'm having.

I'm still having the same issue.

I guess I'm off to start digging into esxi.

Thanks to all who tried to help me :)

Reporting in to say I get the same errors. However, Pimox continues to function for me, though maybe less than optimally. I have a cluster of six. They work 99% of the time, but I've had to rebuild one node multiple times because it stops communicating after a reboot. May or may not be related to the error.