software-layer Map tests to software builds

The tests from the EESSI test suite that are run should depend on which software was built. This PR implements that functionality. Roughly speaking, it does the following:

Get the new modules from module_files.list.txt (in the tempdir)
Use yaml config file (software_to_test.yml) that maps regular expressions matching the (new) module names to a (list of) test(s) that need to be ran for that particular module name. First match in the yml file will be returned, so order is important.
A python code imports both the module_files.list.txt and software_to_test.yml, loops over all build software, each time executing the regular expression matches on the content of the software_to_test.yml, thus finding the tests that should be run for that particular module. If that test wasn't part of an aggregate list yet, it'll append it. Finally, the list is turned into a ReFrame argument string of the form e.g. -n EESSI_GROMACS -n EESSI_ESPRESSO, ... etc.
This argument list will be passed to the ReFrame command in the test step (TODO)

Aug 19 '24 09:08 casparvl

Instance eessi-bot-mc-aws is configured to build for:

architectures: x86_64/generic, x86_64/intel/haswell, x86_64/intel/skylake_avx512, x86_64/amd/zen2, x86_64/amd/zen3, aarch64/generic, aarch64/neoverse_n1, aarch64/neoverse_v1
repositories: eessi.io-2023.06-compat, eessi-hpc.org-2023.06-software, eessi-hpc.org-2023.06-compat, eessi.io-2023.06-software

Aug 19 '24 09:08 eessi-bot[bot]

Instance eessi-bot-mc-azure is configured to build for:

architectures: x86_64/amd/zen4
repositories: eessi.io-2023.06-compat, eessi-hpc.org-2023.06-compat, eessi-hpc.org-2023.06-software, eessi.io-2023.06-software

Aug 19 '24 09:08 eessi-bot[bot]

bot: build repo:eessi.io-2023.06-software arch:zen3

Aug 19 '24 10:08 casparvl

Updates by the bot instance eessi-bot-mc-aws (click for details)

received bot command build repo:eessi.io-2023.06-software arch:zen3 from casparvl
- expanded format: build repository:eessi.io-2023.06-software architecture:zen3
handling command build repository:eessi.io-2023.06-software architecture:zen3 resulted in:
- submitted job 16528, for details & status see https://github.com/EESSI/software-layer/pull/673#issuecomment-2296259476

Aug 19 '24 10:08 eessi-bot[bot]

Updates by the bot instance eessi-bot-mc-azure (click for details)

received bot command build repo:eessi.io-2023.06-software arch:zen3 from casparvl
- expanded format: build repository:eessi.io-2023.06-software architecture:zen3
handling command build repository:eessi.io-2023.06-software architecture:zen3 resulted in:
- no jobs were submitted

Aug 19 '24 10:08 eessi-bot[bot]

New job on instance eessi-bot-mc-aws for architecture x86_64-amd-zen3 for repository eessi.io-2023.06-software in job dir /project/def-users/SHARED/jobs/2024.08/pr_673/16528

date	job status	comment
Aug 19 10:39:11 UTC 2024	submitted	job id `16528` awaits release by job manager
Aug 19 10:39:54 UTC 2024	released	job awaits launch by Slurm scheduler
Aug 19 10:40:55 UTC 2024	running	job `16528` is running
Aug 19 10:57:12 UTC 2024	finished	:cry: FAILURE (click triangle for details) Details :white_check_mark: job output file `slurm-16528.out` :white_check_mark: no message matching `ERROR:` :white_check_mark: no message matching `FAILED:` :white_check_mark: no message matching `required modules missing:` :white_check_mark: found message(s) matching `No missing installations` :white_check_mark: found message matching `.tar.gz created!` Artefacts No artefacts were created or found.
Aug 19 10:57:12 UTC 2024	test result	:grin: SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 18/18 test case(s) from 18 check(s) (0 failure(s), 0 skipped, 0 aborted) Details :white_check_mark: job output file `slurm-16528.out` :white_check_mark: no message matching `ERROR:` :white_check_mark: no message matching `[\sFAILED\s].Ran . test case`

Aug 19 '24 10:08 eessi-bot[bot]

Hmmm... I guess dumping the module_file.list.txt in the current directory in the tarball step doesn't make it available in the test step. Maybe the working dirs are different. I'll try interactively to see what is going on.

ESC[32mSuccesfully ran 'reframe --version'ESC[0m
Loaded mappings from 'tests/eessi_test_mapping/software_to_tests.yml'
Traceback (most recent call last):
  File "/project/60006/SHARED/jobs/2024.08/pr_673/event_422aba90-5e17-11ef-9169-227fc6b2fccc/run_000/linux_x86_64_amd_zen3/eessi.io-2023.06-software/tests/eessi_test_mapping/map_software_to_test.py", line 78, in <module>
    main(args.mapping_file, args.module_list, args.debug)
  File "/project/60006/SHARED/jobs/2024.08/pr_673/event_422aba90-5e17-11ef-9169-227fc6b2fccc/run_000/linux_x86_64_amd_zen3/eessi.io-2023.06-software/tests/eessi_test_mapping/map_software_to_test.py", line 42, in main
    software_names = read_software_names(module_file)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/60006/SHARED/jobs/2024.08/pr_673/event_422aba90-5e17-11ef-9169-227fc6b2fccc/run_000/linux_x86_64_amd_zen3/eessi.io-2023.06-software/tests/eessi_test_mapping/map_software_to_test.py", line 17, in read_software_names
    raise FileNotFoundError(f"Error: {file_path} does not exist.")
FileNotFoundError: Error: module_file.list.txt does not exist.

Aug 19 '24 12:08 casparvl

bot: build repo:eessi.io-2023.06-software arch:zen3

Aug 19 '24 13:08 casparvl

Updates by the bot instance eessi-bot-mc-azure (click for details)

received bot command build repo:eessi.io-2023.06-software arch:zen3 from casparvl
- expanded format: build repository:eessi.io-2023.06-software architecture:zen3
handling command build repository:eessi.io-2023.06-software architecture:zen3 resulted in:
- no jobs were submitted