RPi4 icon indicating copy to clipboard operation
RPi4 copied to clipboard

Version 1.29 No longer boots raspberrypi kernel, only boots upstream kernel

Open Barborica-Alexandru opened this issue 3 years ago • 15 comments

Hello, I upgraded today to 1.29 and when trying to boot a manually compiled kernel from https://github.com/raspberrypi/linux I am greeted by this message: image

However the standard upstream kernel provided by fedora boots just fine. Also previous versions of the firmware including 1.28 boots the compiled kernels just fine.

Barborica-Alexandru avatar Aug 24 '21 20:08 Barborica-Alexandru

Are you booting in DeviceTree or ACPI mode?

samerhaj avatar Aug 24 '21 20:08 samerhaj

This is on ACPI with 3GB limit enforced: image

And this is on pure DeviceTree with 3GB limit enforced: image

Seems the same. Could be possible DeviceTree is not working?

If i grep the configs of the 2 kernels for the word ALIGN the only differences I get are: manually compiled kernel: image fedora provided kernel: image

Here are the 2 config files: configfedora.txt config.txt

Barborica-Alexandru avatar Aug 24 '21 20:08 Barborica-Alexandru

CONFIG_SND_HDA_ALIGNED_MMIO=y being missing from the manually compiled kernel might be problematic because the pi4 cannot do unaligned reads and writes without handwritten assembly workarounds. The compiled kernel is probably using unimplemented/erroneous LDUR/STUR instructions and fails to boot.

TheMindVirus avatar Aug 25 '21 03:08 TheMindVirus

CONFIG_SND_HDA_ALIGNED_MMIO=y being missing from the manually compiled kernel might be problematic because the pi4 cannot do unaligned reads and writes without handwritten assembly workarounds. The compiled kernel is probably using unimplemented/erroneous LDUR/STUR instructions and fails to boot.

But that flag seems to be related to the intel HDA driver which is not present on the raspberrypi. Anyway I am going to experiment with the flags between the kernels and see what makes it boot. However I think this is a regression since there were no problems up to version 1.28

Barborica-Alexandru avatar Aug 25 '21 04:08 Barborica-Alexandru

Despite there being no Intel HDA Sound on Raspberry Pi Hardware, the Linux Kernel for it still contains Intel HDA Drivers/Modules in the tree. This could still be one of many potential causes of the boot failure, the others are listed in the changelog for 1.29: Added Secure Boot, Enabled Boot Discovery Policy and Removed "unnecessary" ACPI CID's.

TheMindVirus avatar Aug 25 '21 05:08 TheMindVirus

I too experience this issue with 5.13 and 5.14 and acpi + device tree. Booting with u-boot and 5.14, also has this issue, so I would guess this is more related to some change in the kernel code than uefi code change.

0n0w1c avatar Aug 25 '21 13:08 0n0w1c

I too experience this issue with 5.13 and 5.14 and acpi + device tree. Booting with u-boot and 5.14, also has this issue, so I would guess this is more related to some change in the kernel code than uefi code change.

It does seem weird that version 1.28 works just fine though. In any case I started from the working config for the fedora mainline kernel and am working my way down to the raspberrypi kernel. This is my current progress, I will update the file as I progress and isolate the flag that makes the kernel boot: config5.13.12.txt config5.10.60.txt

And this is the default bcm2711_defconfig configbroadcomdef.txt

Barborica-Alexandru avatar Aug 25 '21 17:08 Barborica-Alexandru

Please bear in mind that with each version of the UEFI firmware we release, we also update to the latest start4.elf and fixup4.dat from https://github.com/raspberrypi/firmware/tree/master/boot (i.e. from the Raspberry Pi Foundation), and it's not uncommon to see regression being introduced there.

If you want to determine whether the issue has to do with the UEFI firmware or the Raspberry Pi Foundation support files, you should try to replace start4.elf and fixup4.dat of the 1.29 UEFI archive with the ones from the 1.28 version and see what happens.

If it works, then the problem is not with the UEFI firmware.

pbatard avatar Aug 25 '21 17:08 pbatard

Please bear in mind that with each version of the UEFI firmware we release, we also update to the latest start4.elf and fixup4.dat from https://github.com/raspberrypi/firmware/tree/master/boot (i.e. from the Raspberry Pi Foundation), and it's not uncommon to see regression being introduced there.

If you want to determine whether the issue has to do with the UEFI firmware or the Raspberry Pi Foundation support files, you should try to replace start4.elf and fixup4.dat of the 1.29 UEFI archive with the ones from the 1.28 version and see what happens.

If it works, then the problem is not with the UEFI firmware.

Yes that works so it is a regression in the support files.

Barborica-Alexandru avatar Aug 25 '21 17:08 Barborica-Alexandru

I too experience this issue with 5.13 and 5.14 and acpi + device tree. Booting with u-boot and 5.14, also has this issue, so I would guess this is more related to some change in the kernel code than uefi code change.

There appears to be a general issue with XHCI and DeviceTree in recent FW.

In my limited testing (on RPi4 8GB) using 1.29 FW, I confirmed that Fedora 34 IoT works, but OpenSUSE Leap 15.3, and Debian 10 do not. The OS is not seeing XHCI/USB devices. You can confirm with the OS kernel messages on the serial console, which show xhci probe/init errors. .

@jlinton may have more info. He is chasing and has opened a bug against the RPi firmware: https://github.com/raspberrypi/firmware/issues/1617 . He also sent a patch to the EDK2 list (https://edk2.groups.io/g/devel/topic/84933264#79387), but that may not be sufficient if RPi foundation FW/DT files need to change

samerhaj avatar Aug 25 '21 18:08 samerhaj

This is possibly duplicate of https://github.com/pftf/RPi4/issues/157

samerhaj avatar Aug 26 '21 14:08 samerhaj

This is possibly duplicate of #157

It might be related but in my case v1.29 fails to boot kernel 5.10.60 which might not contain these XHCI changes or whatever is causing this.

While trimming down the fedora kernel, after changing some USB flags I did indeed run into this, so yes the problem is probably the same. Screenshot 2021-08-26 231436

Barborica-Alexandru avatar Aug 26 '21 15:08 Barborica-Alexandru

Try it with the 1.32 firmware.

jlinton avatar Oct 23 '21 00:10 jlinton

seems to work fine. Then again I had already migrated to a custom kernel that worked anyway.

Barborica-Alexandru avatar Oct 26 '21 16:10 Barborica-Alexandru

This is closed? Or is the associated #157 (and RPi3 #37) still an open issue as well?

paulwratt avatar Feb 17 '22 23:02 paulwratt

I'm going to close this since it's been so long since I ran into this issue that I remember nothing about it.

Barborica-Alexandru avatar Mar 13 '23 23:03 Barborica-Alexandru