Introduction of board and SoC scheme v2.

[tejlmand]

This PR introduces a board and SoC scheme v2.

The benefits of board and SoC scheme v2 are:

• Inherit BOARD setting from CMake
  • https://github.com/zephyrproject-rtos/zephyr/pull/49260#issuecomment-1220381626
  • https://github.com/zephyrproject-rtos/zephyr/pull/44654#discussion_r909335964
  • This aligns how BOARD is passed from CMake --> Kconfig to the way BOARD_REVISION is handled
  • this removes any inconsistency in names
  • Removes the need to keep names in sync, and updating them in two places.
• Remove the need for prompts on SoCs
  • In zephyr/main today then the SoC is set in a config file instead of being selected directly by the board. This can be confusing to new-comers, as they might wonder if an SoC can be changed because all SoCs in Zephyr appears in a choice menu in menuconfig Changing an SoC without a pristine build is not supported or tested, so no reason to expose such options directly to users.
• Board v2 uses select of SoC instead of a configuration file
  • This makes it clearer and conforming to rest of Kconfig tree, where a given setting will generally select other settings that it requires and not expect all those settings being defined in config files.
• The SoC v2 Kconfig tree has specific rules that can be described and thus tested in compliance (pending task)
• The SoC v2 Kconfig tree is independent, allowing it to be re-used, for example by sysbuild
  • https://github.com/zephyrproject-rtos/zephyr/issues/43488.

Note, there is no requirement for existing boards or SoCs to be updated to v2, but if a board and SoC want to benefit from above, then they can of course be updated.

• [x] Create board v2 infrastructure in CMake and Kconfig
• [x] Create SoC v2 infrastructure in CMake and Kconfig
• [ ] Document board v2 requirements
• [ ] Document SoC v2 requirements
• [x] Update compliance check to verify a board / SoC v2 conforms to the rules

Draft PR until all above requirements has been implemented.

[andyross]

Still outside my area of expertise, but this scheme I like much better than N-thousand-line PRs!

[tejlmand]

@tejlmand it would be nice if "board scheme v2" didn't require a boards/ARCH/boardname hierarchy so that we wouldn't have to hack around the zephyr file system standard for targets with multiple architectures on a single PCB.

A very good point indeed, and I assume same comment applies to SoC tree. At first I tried to stay close to current scheme, but with the v2, reconsidering the arch hierarchy would make a lot of sense.

To others reading this, please give a 👍 if you think scheme v2 should actually get rid of ARCH as suggested by @mbolivar-nordic.

This would especially make sense for boards with multiple SoCs of different archs, or multicore SoCs where different cores might be of different archs.

[mbolivar-nordic]

I assume same comment applies to SoC tree.

It does.

[nashif]

for those who have been long enough here, you will remember the days we had to pass ARCH and BOARD to make to build a board. I personally have no issues with that, although it was always easier to find the boards I was looking for, given that some of us are invested in more than one architecture and not necessarily ARM. Let me undersdtand though, how is that going to change anything for multi-arch platforms? I mean both the SOC and BOARD would still be tied to an architecture, is the intention here to have a 2 cores of different architectures in the same silicon be on the same level? Is it only about having one level less and a shorter path or are you thinking about something else supporting this type of hardware?

[mbolivar-nordic]

both the SOC and BOARD would still be tied to an architecture

No, that's what I'm saying should be changed for the v2 hardware scheme. We have known this has been a problem for years:

https://github.com/zephyrproject-rtos/zephyr/commit/0d811b9aee6a6410f73fdfb4d4072d4d16b05a52#diff-cc856e9b347fddf6d5a472d1330e1e71e805aee2e66a48b9251b528af7f390f8R290

Decoupling "board" and "soc" from "arch" is exactly what I am suggesting. An SoC can have CPU clusters with as many different architectures as the vendor wants to cram in there, so any target board can too.

[mbolivar-nordic]

0d811b9#diff-cc856e9b347fddf6d5a472d1330e1e71e805aee2e66a48b9251b528af7f390f8R290

Link is not working well; I'm trying to reference this part of the old getting started:

.. [#board_misnomer]
   This has become something of a misnomer over time. While the target can be,
   and often is, a microprocessor running on its own dedicated hardware
   board, Zephyr also supports using QEMU to run targets built for other
   architectures in emulation, targets which produce native host system
   binaries that implement Zephyr's driver interfaces with POSIX APIs, and even
   running different Zephyr-based binaries on CPU cores of differing
   architectures on the same physical chip. Each of these hardware
   configurations is called a "board," even though that doesn't always make
   perfect sense in context.

[tejlmand]

Link is not working well; I'm trying to reference this part of the old getting started:

Maybe use this link to the old docs instead: https://docs.zephyrproject.org/1.14.1/getting_started/index.html#board-misnomer

[tejlmand]

Decoupling "board" and "soc" from "arch" is exactly what I am suggesting. An SoC can have CPU clusters with as many different architectures as the vendor wants to cram in there, so any target board can too.

@mbolivar-nordic While i'm agreeing a lot with your statement, I also like the structure of multiple levels, as a form of filtering. This means a user can look only at arm, riscv, x86 based boards.

Have you considered if everything should be at top-level or we should have another structure ? Vendor based, like boards/intel/<intel-boards>. That may make sense for some vendors, but we risk n-vendor folders just containing a single board, which also leads to mess. And what if you're having a board with an intel chip, but from vendor b, then in some cases it could be nicer to filter on soc-vendor rather than board vendor.

One requirement for the v2 scheme is to be self-contained. A board selects a SoC / SoC core, and a SoC core selects the architecture (done indirectly today through the folder hierarchy).

Now, because the v2 scheme must be self-contained, then that also means we could easily extend west boards, so that we simply build the board / SoC / arch Kconfig tree, and thus can be able to support: west boards --> list all boards west boards --arch arm --> list all boards where the has a SoC, where as minimum one core is arm based. west boards --vendor intel --> list all boards where the board vendor is intel. west boards --soc-vendor nordic --> list all boards where the soc vendor is nordic.

Such a possibility would imo make it acceptable to have all boards and SoCs at a single level, as there is now an option to easily filter boards.

Note 1: This was not an original use-case, but a use-case that will be possible in the v2 scheme, but I will need to take a look at the arch v2 scheme, as that has not initially been part of this work.

Note 2: Today addressing a specific core on a SoC is defined as a independent board, like nrf5340dk_nrf5340_cpuapp vs. nrf5340dk_nrf5340_cpuapp. Should we consider addressing this in scheme v2 as well. Formalize how cores are addressed, for example as: nrf5340dk_nrf5340@app and nrf5340dk_nrf5340@net And then a west boards --vendor nordic could present:

$ west boards --vendor nordic
...
board: nrf52840dk_nrf52840, SoC: nrf52840 (1 core): core 0: arm
board: nrf5340dk_nrf5340,   SoC: nrf5340 (2 cores): core 0: arm, core 1: arm
...

[tejlmand]

Update compliance check to verify a SoC v2 conforms to the rules

This is tested here, #50356, with an invalid defined board v2, and the result:

warning: undefined symbol NATIVE_APPLICATION:

- Referenced at boards/arm/nrf52840dk_nrf52840/Kconfig.v2.nrf52840dk_nrf52840:4:

config BOARD_NRF52840DK_NRF52840
	bool
	default y
	select SOC_SERIES_NRF52X
	select SOC_NRF52840_QIAA
	select NATIVE_APPLICATION
ERROR   : Test KconfigBoardV2 failed: Undefined Kconfig symbols:

warning: undefined symbol NATIVE_APPLICATION:

[nashif]

An SoC can have CPU clusters with as many different architectures as the vendor wants to cram in there, so any target board can too.

in zephyr we are using SoC in the wrong way. The way it is now, it represents just one IP that is tied to one architecture. An SoC with two different IP blocks with different architectures would be a superset in this case, a system.... If we are looking at craming multiple IPs under SoC to represent a system, we might be looking at more than just a change to Kconfig..

[tejlmand]

Toolchain WG summary:

• Current structure makes it confusing to users on multi-soc and multi-core-soc boards
  • example: Up Squared which has multi-core SoC (Celeron / Pentium core (x86) + audio DSP (xtensa)) and altera FPGA MAX10 SoC.
  • nRF9160dk contains both a nRF9160 and a nRF52840 SoC (same arch)
• The current structure also results in build system needing to setup a generic toolchain for dts pre-processing and later switch the toolchain to a target toolchain. This is because arch (and thus toolchain arch) is not known until after DTS. A v2 scheme could benefit the build system, so that board, SoC, and arch can be determined before DTS invocation, allowing build system to decide toolchain early
  • This has the extra benefit of allowing further cleanup of cmake/modules/*.cmake in areas such as:
    • toolchain handling
    • arch and soc CMake

[nashif]

thought more about all of this. In the ideal world and looking down the path of supporting products with multi-arch SoCs, an SoC (as we have it today) would be just contained and part of a Multiprocessor system on a chip (mSoC) and we would have one board definition that builds all parts of the mSoC and generates artifacts for those using different toolchains if applicable (basically what sysbuild does) in a multi-stage build process.....

[tejlmand]

would be just contained and part of a Multiprocessor system on a chip (mSoC) and we would have one board definition that builds all parts of the mSoC and generates artifacts for those using different toolchains if applicable (basically what sysbuild does) in a multi-stage build process.....

correct, and i'm trying to design this, including thoughts on scalability.

We basically want a common tree (to make maintaining boards / SoC easier and cleaner), and in the highlevel system (sysbuild) be able to specify just the board (system).

The highlevel (sysbuild) will then pass specific information down to the individual build stages, for example build Zephyr sample N with toolchain X, for core A, build sample M, toolchain Y, core B, etc.

But it should still be possible to directly request a build of sample N, toolchain X, core A. Also each build of Zephyr must still be able to identify exactly for which arch / core its being built, as to ensure correct settings (and defaults) are available.

Therefore we want common knowledge shared in the tree, and not scattered around.

[mbolivar-nordic]

would be just contained and part of a Multiprocessor system on a chip (mSoC) and we would have one board definition that builds all parts of the mSoC and generates artifacts for those using different toolchains if applicable (basically what sysbuild does) in a multi-stage build process.....

correct, and i'm trying to design this, including thoughts on scalability.

... and I'm trying to help on the devicetree side using system devicetree, which I'll send an RFC about soon.

[mbolivar-nordic]

On Mon, Sep 19 2022, Torsten Tejlmand Rasmussen wrote:

Decoupling "board" and "soc" from "arch" is exactly what I am suggesting. An SoC can have CPU clusters with as many different architectures as the vendor wants to cram in there, so any target board can too.

@mbolivar-nordic While i'm agreeing a lot with your statement, I also like the structure of multiple levels, as a form of filtering. This means a user can look only at arm, riscv, x86 based boards.

Have you considered if everything should be at top-level or we should have another structure ? Vendor based, like boards/intel/<intel-boards>. That may make sense for some vendors, but we risk n-vendor folders just containing a single board, which also leads to mess.

I agree, it's a tricky problem to solve. I just think we should not enforce this false ARCH-based hierarchy in v2.

[yvanderv]

On Mon, Sep 19 2022, Torsten Tejlmand Rasmussen wrote: > Decoupling "board" and "soc" from "arch" is exactly what I am suggesting. An SoC can have CPU clusters with as many different architectures as the vendor wants to cram in there, so any target board can too. @mbolivar-nordic While i'm agreeing a lot with your statement, I also like the structure of multiple levels, as a form of filtering. This means a user can look only at arm, riscv, x86 based boards. Have you considered if everything should be at top-level or we should have another structure ? Vendor based, like boards/intel/<intel-boards>. That may make sense for some vendors, but we risk n-vendor folders just containing a single board, which also leads to mess. I agree, it's a tricky problem to solve. I just think we should not enforce this false ARCH-based hierarchy in v2.

Under the assumption that a board may have multiple architectures, such as an MCU and a DSP, and Zephyr was ported to both. How would I build then? west build -b -s ... ? Or under this scheme we have two different "boards", one from the point of view of the MCU and DSP? How building for the whole system?

One use case is to indeed filter out the list boards by arch, soc-vendor, board-vendor. twister could then be asked to build for all boards of a certain *vendor, etc. Another use case is of course to just run a lightweight build of a sample for all boards of a certain 'kind', something like: $ for board in $(west boards --board-vendor super-extra) ; do west build -b ${board} -.... sample/some done It is much faster than running twister to build.

[mbolivar-nordic]

How would I build then? west build -b -s ... ? Or under this scheme we have two different "boards", one from the point of view of the MCU and DSP? How building for the whole system?

TBD, but for context, this is all part of a much larger issue that is intended to address your question: https://github.com/zephyrproject-rtos/zephyr/issues/51833

[github-actions[bot]]

This pull request has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this pull request will automatically be closed in 14 days. Note, that you can always re-open a closed pull request at any time.

[nordicjm]

Rebased: https://github.com/nordicjm/zephyr/tree/boardv2rework

[nordicjm]

@mbolivar-nordic @nashif @andyross @galak when time allows, can you give this a look over and see if you think this https://github.com/nordicjm/zephyr/tree/boardv2rework and a test conversion of a board in https://github.com/zephyrproject-rtos/zephyr/pull/50356 (look at https://github.com/nordicjm/zephyr/tree/boardv2reworknrf52 for the commits) are acceptable? Would like to aim to get this into a finished state so it can be included in zephyr 3.4, and will probably need to bring it up at the architecture working group sometime in the future @carlescufi

[github-actions[bot]]

This pull request has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this pull request will automatically be closed in 14 days. Note, that you can always re-open a closed pull request at any time.

[tejlmand]

This PR needs documentation of the new scheme, a migration guide, i.e. describe how to move a board/soc from v1 to v2

Correct, and that is also listed in the PR description as outstanding. (Just noticed the DNM was not applied when moved out of draft, so added that label now) Docs are being prepared.

some examples of converted boards of different complexity to show how it works and that it covers all corner cases, i.e. simple soc/board combo, multi core of same arch, multi core of different architectures etc.

I have a branch for transitioning the Nordic nRF series SoC and boards here: https://github.com/tejlmand/zephyr/tree/nrf52_adopt_soc_scheme_v2/ but I initially kept that work outside the PR, as to:

1. show that the introduction of HWMv2 doesn't break any existing board (meaning migration can be done per SoC)
2. Minimize the change set itself that reviewers would need to consider.

But I have now decided to migrate two SoCs and corresponding boards and include this migration in the PR itself, so that reviewers can have an idea about the amount of work needed.

[nashif]

How much of the transition from v1 to v2 can be scripted to help with mass migration of boards and SoCs? What are the key spots where manual intevention and review will be needed?

[tejlmand]

How much of the transition from v1 to v2 can be scripted to help with mass migration of boards and SoCs? What are the key spots where manual intevention and review will be needed?

There are some common patterns which should be scriptable, but one thing that remains to decide is if we want to include SoC name in the board / make it mandatory to specify.

[carlescufi]

Architecture WG:

• Required meta-files with a list of all boards and socs are not received well: generate the data instead from having searched recursively for board.yml and soc.yml
• Discussion about variants and whether they represent software configuration or hardware variants
• @nashif asks whether a variant could be set by default so that building without specifying the variant works without sysbuild
• Objection to Kconfig.zephyr and Kconfig.zephyr.defconfig having zephyr in the name
• Transition plan: participants prefer a quick transition instead of one expanding over time

[carlescufi]

Arch WG:

• @tejlmand describes the state of the PR
• @nashif mentions that list_hwm.py is confusing, list_hardware.py would likely be more understandable
• @nashif states that we need to reduce the complexity of those yaml files (cpucluster, variant, ...). We need to make sure that is future-proof and clear enough for users
• @galak suggest having a script that can generate and verify soc.yml from Devicetree file, and more importantly verify that the Devicetree and then .yml never go out of sync
• @galak also suggests to have the schemas for soc.yml and board.yml as standalone YAML files. @tejlmand and @carlescufi agree
• In agreement that transitioning boards should happen as fast as possible, and we agree to use a collaboration branch

[tejlmand]

lgtm, only a few minor suggestions. I'd personally avoid the "hwmv2" or "v2" as much as possible, considering the new model will quickly replace "v1". For example, having new boards/socs in a "v2/" folder will imply moving them back soon. Also, all cmake, python scripts etc. will need to be cleaned up.

Thanks for the feedback, and yes, the hwmv2 / v2 should be short lived, and with the wish for quick transition even shorter lived than initial anticipated, which is very nice.

However, there are also oot users with custom arch / SoC / boards in current format, so in some places I think we still need to differentiate between v1 and v2.

Of course, if a SoC is moved to v2, then boards using that SoC must be updated, but an oot SoC (and boards using such SoC) may not be updated immediately.

[tejlmand]

(above issues are things that need resolving before merging to main, fine to merge this to collab branch as-is and fix them in that branch later)

Marked issues resolved as we are tracking open issues which must be resolved before merging to main here: https://github.com/zephyrproject-rtos/zephyr/issues/51831#issue-1430903882

[nashif]

Wondering if it would make sense to move the v2 structures for both boards and socs into a completely new folder and drop the v2 naming completely, this way the transition happens only one time and we do not end up with a everything moved to boards/v2 and socs/v2 to later have to remove all of this again once v1 is deprecated, so for example this could be a new top level folder, call it hw, platforms, ports or whatever and under there we could have both soc and boards, so

hw (or whatever)
├── boards
└── socs

IMO this is more future proof and easy to understand and traverse, having 'v2' in there and mixing old v1 and v2 in same strucutre just looks messy, let alone the fact that we will have to remove the v2 naming at some point.

This request should not block this from going into the collab branch and continue the progress, but it is something to consider IMO.

[nashif]

west boards   | grep mps3
mps3

then...

i9:zephyr(board_soc_scheme_v2): west build -b mps3 samples/hello_world
-- west build: making build dir /home/nashif/zephyrproject/zephyr/build pristine
-- west build: generating a build system
Loading Zephyr default modules (Zephyr base).
-- Application: /home/nashif/zephyrproject/zephyr/samples/hello_world
-- CMake version: 3.21.1
-- Found Python3: /usr/bin/python (found suitable version "3.11.6", minimum required is "3.8") found components: Interpreter
-- Cache files will be written to: /home/nashif/.cache/zephyr
-- Zephyr version: 3.5.99 (/home/nashif/zephyrproject/zephyr)
-- Found west (found suitable version "1.2.0", minimum required is "0.14.0")
CMake Error at /home/nashif/zephyrproject/zephyr/cmake/modules/boards.cmake:223 (message):
  Board identifier `` for board `mps3` not found.  Please specify a valid
  board.

  Valid board identifiers for mps3 are:

  mps3/an547

  mps3/an547/ns

shouldnt the boards be listed as above instead of mps3?