build-system: auto-locate modules by name

[maribu]

Contribution description

This PR introduces the MODULE_DIRS variable that works just like EXTERNAL_MODULE_DIRS, but it is intended for internal modules. The major difference is that external modules can have the same name as an internal module, in which case the external module is used instead of the internal one. This allows to provide drop-in replacements of internal modules, which can be useful during development e.g. to quickly benchmark different implementations.

Testing procedure

All applications for all boards should still build and yield identical binaries.

Issues/PRs references

Follow up of https://github.com/RIOT-OS/RIOT/pull/16104

[fjmolinas]

This only works with modules in a directory matching the module name right?

[maribu]

This only works with modules in a directory matching the module name right?

Exactly. For those modules, one still has to manually add the dirs to the DIRS variable. But e.g. for all modules in drivers, this already works.

[fjmolinas]

This only works with modules in a directory matching the module name right?

Exactly. For those modules, one still has to manually add the dirs to the DIRS variable. But e.g. for all modules in drivers, this already works.

Can this be documented?

[fjmolinas]

This only works with modules in a directory matching the module name right?

Exactly. For those modules, one still has to manually add the dirs to the DIRS variable. But e.g. for all modules in drivers, this already works.

Can this be documented?

BTW, couldn't we do the same for external modules as well?

[maribu]

This only works with modules in a directory matching the module name right?

Exactly. For those modules, one still has to manually add the dirs to the DIRS variable. But e.g. for all modules in drivers, this already works.

Can this be documented?

BTW, couldn't we do the same for external modules as well?

Yes, but at least one external module has to be found without, which could extend the DIRS variable as needed. (Or the applications Makefile could do so.)

I intend btw. to extend the mechanism to allow assigning prefixes to search dirs. E.g sys/gnrc should find module gnrc_%, where % is the folder name. With this, we might be able to automatically locate all modules eventually.

[MrKevinWeiss]

Do you think this should be included in this coming release or should we push it?

[benpicco]

Needs a rebase (feel free to squash while you're at it)

[maribu]

Module names that mix the directories into it are a common practice in the build system, e.g.: foo/bar and MODULE = foo_bar.

That should actually be possible to still auto-detect, as long as the pattern is consistent. I have an idea how this might be possible to implement in make without complexity getting too high.

[OlegHahm]

@maribu, ping!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

[maribu]

Let's close this. It is so outdated that one would need to start from scratch anyway. And maybe after the KConfig migration we can switch to a better build system instead :)