RIOT-OS
boards: introduce board tiers
Contribution description
The idea is to provide an infrastructure that allows selecting pre-defined subsets on which to run CI compilation tests. The following tiers are specified:
- tier 0:
- only 5 boards, for super fast CI runs
- based on popularity and heterogeneity of the MCUs used
- tier 1:
- one board for every MCU family, preferring the MCU variant with most peripherals / RAM / flash where possible
- tier 2:
- all boards
A tier 0 board is also included in tier 1 and tier 2, a tier 1 board is included in tier 2. All tier are implemented as features. Hence, a FEATURES_REQUIRED += board_tier_0 selects only 5 boards, a FEATURES_REQUIRED += board_tier_1 selects one board per MCU family (also including all tier 0 boards), and finally FEATURES_REQUIRED += board_tier_3 is fulfilled by all boards.
The idea is to add labels to PRs that will export an FEATURES_REQUIRED environment variable to limit the result of make info-boards-supported to the given set. Maintainers can then decide on the level of CI coverage a PR should get, depending on how scary the code looks.
Testing procedure
The CI should be able to detect that feature modeling is consistent with both TEST_KCONFIG=1 and TEST_KCONFIG=0. Otherwise, proof-reading is the only way to test this.
Issues/PRs references
None
I wonder if the features are the best place for this? I would think this is more a CI specific thing. Maybe this would be better to have in makefile.ci instead? This also means we don't have to worry about dependency modeling. We can also default to tier 2 meaning only tier 1 and tier 2 boards need to be flagged?
We can also default to tier 2 meaning only tier 1 and tier 2 boards need to be flagged?
It is implemented like that :)
Maybe this would be better to have in makefile.ci instead?
Maybe. But note that features do implement already exactly the kind of mechanism this needs: Filtering only boards that have a magic flag set. I try to avoid adding new stuff to our build system, as the current level of its complexity already gives me nightmares. With features, it is trivial to figure out which boards to build a given app for
FEATURES_REQUIRED=board_tier_1 make -C tests/<FOO_TEST> info-board-supported
I was hoping that this could also be used outside of the CI. E.g. when I implement a new feature, I typically add a test app to this. It should be more than possible to locally build a single app for all tier 1 boards, but it will already find 90% of the issues Murdock will uncover.
I think this is a great step. Having a central list (that's what @MrKevinWeiss proposed?) would have pros and cons... If there's no opposition, I'd say let's go with this and try! So please rebase & squash.
Murdock results
:heavy_check_mark: PASSED
2a69358c0968fedc913b212d279725fc19d39dbe boards: introduce board tiers
| Success | Failures | Total | Runtime |
|---|---|---|---|
| 6770 | 0 | 6770 | 13m:53s |
Artifacts
Since we already have this now, is this PR still needed? Or would it be cleaner to switch Murdock to use this information?
