coreboot Power9/Power10 support

[tlaurion]

https://doc.coreboot.org/contributing/project_ideas.html#support-power9-power8-in-coreboot

Hardware provided by RaptorEngineering

• RYF board: Talos II

[tlaurion]

Interests to have Open Source Hardware and Compartmentalization for the future of computing:

• [ ] Libre RISC-V -> Power based free GPU
• [ ] QubesOS PPC64 support

@madscientist159

[tlaurion]

@pietrushnic :)

[miczyg1]

@3mdeb is evaluating the effort of porting the POWER9 Talos II and Talos II Lite to coreboot. The plan should be announced soon.

[tlaurion]

Underway!

[macpijan]

@tlaurion As you know the coreboot effort is progressing (https://github.com/3mdeb/openpower-coreboot-docs/tree/main/releases, https://github.com/3mdeb/coreboot/tree/talos_2_support_payload).

I believe we are at the point where we could start working on heads, so we can start testing faster once the coreboot bits are in place. I can see following tasks:

1. Review kernel patches from skiroot

• Review kernel patches in skiroot: https://git.raptorcs.com/git/talos-op-build/tree/openpower/linux
• Review the tree (https://git.raptorcs.com/git/talos-op-linux/) if it carries some addiitonal patches. At a first glance, it looks as like a plain linux tree, though.

2. Update Linux

• Use the same version as in: https://git.raptorcs.com/git/talos-op-linux/ for simplicity
• Apply patches from (first task) if necessary. If unsure, better to apply at first.
• Simply add support for the linux we need here (and apply patches if needed): https://github.com/osresearch/heads/blob/master/modules/linux ; preferably this linux version is only used for the talos II target
• we can test it somehow, but it does not need to build at this point yet (we will try to do that in fourth task)

3. Update coreboot - build from 3mdeb's fork for now

• Add the possibility to build coreboot from this branch: https://github.com/3mdeb/coreboot/tree/talos_2_support_payload
• As in the Linux case:
  • https://github.com/osresearch/heads/blob/master/modules/coreboot
  • should be a variant, used only by the Talos II board for now
  • dosn’t need to build correctly yet in this task

4. Add config for Talos II board

• configs are here: https://github.com/osresearch/heads/tree/master/config
  • there is one for coreboot and one for Linux
  • the coreboot config: https://github.com/3mdeb/coreboot/blob/talos_2_support_payload/configs/config.raptor-cs-talos-2
  • the Linux config - ideally find out which is the config used in op-build: https://git.raptorcs.com/git/talos-op-build/tree/ we can also flash the original image and dump config from Linux userspace (e.g. /proc/config.gz if it exists) ; we know that the in-tree powernv_defconfig is the default one for POWER9 and it works ok on our board, so we could use that as well probably
  • there might be some additional requirements heads has on the kernel configuration which are not present in the original config used in the op-build (e.g. drivers for smart cards etc.)

Please let me know if you see anything more. If the plan looks ok, we could split them into issues and track the progress in public here.

[macpijan]

There are two more complex threads as well here.

1. We won't be able to start heads kernel as a coreboot Linux payload as it is currently done for other platforms. At least for now, we only can start skiboot from coreboot. We need it for OPAL. Then, the skiboot can load Linux - in this case it could load heads kernel and initramfs.

2. AFAIK, heads currently build everything in this repo and assembles it into a single ROM file to be flashed in the SPI. In case of the POWER9, we also need to build a single flash image (PNOR). It is, hovewer, much more complex process. Apart from the coreboot and heads stuff, we need a proper layout of this file and some more firmware files (like the OCC firmware which is also briefly mentioned here). The PNOR file and all of the firmware components are built using this repo: https://github.com/open-power/op-build Replicating the PNOR layout creation and building all of the firmware components as part of the heads repo would be quite complex.

Currently, we are reusing the existing build scripts from the op-build repository. We output PNOR file from there and replace Hostboot with coreboot. We propose to do the similar here. So we could build the PNOR file first from the original op-build repository and then replace selected partitions:

• Hostboot with coreboot
• skiroot + petitboot with heads kernel + heads initramfs

There is a tool which allows us to modify the existing PNOR firmware file: https://github.com/3mdeb/openpower-coreboot-docs/blob/main/devnotes/porting.md#ffs-tools

[tlaurion]

@macpijan I can try to do a quick PR draft on which the first points would be addressed as a start.

• modify coreboot module so that a new version points to a specific commit and download and measure the tarball checksums for power.
• modify linux module so that a new version does the same as above, and have the patches available in the patches directory for the same version (git commit) to be applied post decompression prior of building
• create a board configuration which will point to:
  • Put basic power coreboot config in tree
  • Put basic linux config in tree

Would that be helpful as a start? It seems it would ease onboarding to heads ecosystem, while the build will probably fail, of course.

[SergiiDmytruk]

@tlaurion Yes, that would be helpful and speed things up.

[SergiiDmytruk]

I decided to not wait and give it a try and might have addressed basic points already, hope you didn't start yet. My WIP changes are here if you want to take a look and maybe correct obvious mistakes early on. Doesn't build yet (kernel build complains about non-retpoline compiler).

[macpijan]

In regards to Linux config: I believe this one is rather used: https://git.raptorcs.com/git/talos-op-build/tree/openpower/configs/linux

The config dumped from running device: https://github.com/3mdeb/openpower-coreboot-docs/pull/41/files

[tlaurion]

retpoline

@SergiiDmytruk https://github.com/osresearch/heads/commit/268fb90623a49e7645becca456365c23730cb6b1#diff-a890ddf36ae7dca110b70df101c97525a4d8265212ebed3f92562d47162af3a2R322

You can remove support inside of the linux config. Everything looks good at first glance. Not sure if the 5.5 kernel line and patches should be shared across all boards that would depend on it or if the version specified under modules/linux should be power related only?

[SergiiDmytruk]

You can remove support inside of the linux config. Everything looks good at first glance.

Thanks.

Not sure if the 5.5 kernel line and patches should be shared across all boards that would depend on it or if the version specified under modules/linux should be power related only?

I've seen some of those patches applied in 5.6-rc, but others might be power-specific. I'll make this kernel version specific to power.

[SergiiDmytruk]

Limiting to power build didn't work as I expected. This:

else ifeq "$(CONFIG_LINUX_VERSION)" "5.5-openpower"
linux_version := 5.5
linux_hash := a6fbd4ee903c128367892c2393ee0d9657b6ed3ea90016d4dc6f1f6da20b2330

Only suggests that it shouldn't be used for anything else, while patches are still under linux-5.5 directory. I can't change that because then 5.5-openpower version is used to download archive. Also tried to add a separate linux_patch_version, but it won't affect anything under build/. However, that might not be an issue given that you can't mix builds for different architectures anyway (they occupy the same directories). Any thoughts? Maybe I just didn't do it right.

[tlaurion]

Limiting to power build didn't work as I expected. This:

else ifeq "$(CONFIG_LINUX_VERSION)" "5.5-openpower"
linux_version := 5.5
linux_hash := a6fbd4ee903c128367892c2393ee0d9657b6ed3ea90016d4dc6f1f6da20b2330

Only suggests that it shouldn't be used for anything else, while patches are still under linux-5.5 directory. I can't change that because then 5.5-openpower version is used to download archive. Also tried to add a separate linux_patch_version, but it won't affect anything under build/. However, that might not be an issue given that you can't mix builds for different architectures anyway (they occupy the same directories). Any thoughts? Maybe I just didn't do it right.

@SergiiDmytruk Right. Well, the same logic for git/url could be used to have the module depend on ?= by default if not defined and have power linux patch dir and build dir being different then other x86 arch. Not sure what would be the best option here. But once again, if the patches to linux are not breaking x86, maybe they can coexist?

[tlaurion]

Another approach could be to play around https://github.com/SergiiDmytruk/heads/blob/4d9ef955bcc23fb362a4113002afa4fe1545be17/modules/linux#L30

And append the arch to the linux_base_dir. I have not reviewed the patches content, but if the build dirs are different from archs, even if same kernel version, it would result in a different build dir. So no need for mr_proper, which would mean not confict between board builds even on CI.

[SergiiDmytruk]

I have a summary of the patches, they change things in generic files, I'm not sure it's a good idea to use them for x86, maybe they weren't tested there at all:

• 0001 – fix for shutting down xHCI controller
• 0002 – adding extra part to kernel version
• 0003 – DMA controller crash fix
• 0004 – VGA palette fix
• 0005 – one-line correction for default value of 0004's parameter
• 0006-0008 – amdgpu fixes for cards with Navi chipset (they use FPU, which needs extra stuff for Power)

Another approach could be [..]

This is what I used. Frankly, I didn't understand the first one.

So no need for mr_proper, which would mean not confict between board builds even on CI.

For the kernel, yes. However, other modules that are built in-tree (in build/$module/ instead of build/$module/$board) still can't be shared.

I've re-committed changes and made https://github.com/osresearch/heads/pull/1002 hoping for feedback on the changes I had to make to build successfully.

[madscientist159]

@SergiiDmytruk The XHCI patch I believe is to allow the device to survive a kexec(). x86 tends to get around a lot of those types of issues by not kexec()ing in most use cases, whereas POWER kexec()s every time an OS loads.

[tlaurion]

This is now stable under 0.7

---

v0.7.0 - 2023-07-26

Fixed

• [CPU appears to be stuck on initial frequency](https://github.com/Dasharo/dasharo-issues/issues/35)
  

• [No flashrom support](https://github.com/Dasharo/dasharo-issues/issues/190)
  

• [0.6 Release - Cannot boot if no TPM](https://github.com/Dasharo/dasharo-issues/issues/191)
  

• [OS-level access to CBMEM](https://github.com/Dasharo/dasharo-issues/issues/69)
  

• [TPM discovery and usage stability](https://github.com/Dasharo/dasharo-issues/issues/415)
  

• [Dual CPU setup - Second fan at full speed](https://github.com/Dasharo/dasharo-issues/issues/416)
  

Known issues

• [Missing parts of Device Tree describing specific unit (VPD, serial numbers)](https://github.com/Dasharo/dasharo-issues/issues/32)
  

• [No DIMM temperatures reported](https://github.com/Dasharo/dasharo-issues/issues/446)