Ensuring modules can be rebuilt

[ocaisa]

When including software in the EESSI stack, we should not only ensure that the software can be built on the various archs, but also that --module-only returns an identical set of modules as the original installation.

[ocaisa]

This will also allow us to easily perform the sanity checks on different OSes (and help us to avoid issues like #99 )

[ocaisa]

This is something that we could do in our CI since rebuilding the module tree doesn't take too long.

[ocaisa]

--module-only --rebuild could do that for us and will also compare module files (but requires https://github.com/easybuilders/easybuild-framework/pull/3645 in order for the sanity check to run when using this combination)...but because of --robot it won't check any dependencies.

An easy solution is to tell it to build the module tree in a different/temporary path since that will also do all the dependencies. The downside is that you then need to compare the new modules to the old ones in a separate step.

[ocaisa]

@Flamefire has done a good bit of work for this but it is clearly something that is still relevant, indeed I believe we should be shipping the EasyBuildMNS module tree and a hierarchical tree.

[Flamefire]

True. Supporting module-only rebuilds would still be good as some modules encode config options such as PYTHONPATH vs EBPYTHONPATHS and CPATH

However this is hard for 3 reasons I encountered:

• An existing easyconfig (from the reprod folder) generally requires a specific version of EasyBuild or at least easyblocks as behavior might be changed and corresponding easyconfig parameters might be added/changed/removed
• Rebuilding using the easyconfig from another EasyBuild version might fail sanity checks, e.g. Abseil switched from dynamic to static libs and the sanity check expects either or
• Using any EasyBuild version might not work with existing config files/env-variables as EasyBuild options are added/removed/changed and an error will be issues on those. Or defaults might have changed.

So shipping both module trees would indeed help and having reliable module-only rebuilds (in the same configuration/versions at least) is required for that anyway.

[boegel]

@Flamefire has done a good bit of work for this but it is clearly something that is still relevant, indeed I believe we should be shipping the EasyBuildMNS module tree and a hierarchical tree.

And maybe also module files in both Lua & Tcl syntax...