software-layer icon indicating copy to clipboard operation
software-layer copied to clipboard

Add pre-install hook that raises an error in case `keeppreviousinstall` (used for rebuilds) is set but the installation directory still contains files

Open bedroge opened this issue 10 months ago • 6 comments

This is an enhancement for #871 and implements the comment from @boegel in https://github.com/EESSI/software-layer/pull/871#issuecomment-2606523933.

I've tested it locally with EESSI-extend for the following scenarios:

  • no keeppreviousinstall -> this always works, as expected
  • with --try-amend=keeppreviousinstall=True:
    • empty install dir -> works
    • only some empty subdirs in the install dir -> works
    • file in the root of install dir -> error
    • file in a subdir -> error

You can test it by loading EESSI-extend, overriding EASYBUILD_HOOKS and setting it to the hooks file of this PR, and doing a small build with or without --try-amend=keeppreviousinstall=True.

bedroge avatar Jan 23 '25 15:01 bedroge

Instance eessi-bot-mc-aws is configured to build for:

  • architectures: x86_64/generic, x86_64/intel/haswell, x86_64/intel/sapphire_rapids, x86_64/intel/skylake_avx512, x86_64/amd/zen2, x86_64/amd/zen3, aarch64/generic, aarch64/neoverse_n1, aarch64/neoverse_v1
  • repositories: eessi.io-2023.06-software, eessi.io-2023.06-compat

eessi-bot[bot] avatar Jan 23 '25 15:01 eessi-bot[bot]

Instance eessi-bot-mc-azure is configured to build for:

  • architectures: x86_64/amd/zen4
  • repositories: eessi.io-2023.06-software, eessi.io-2023.06-compat

eessi-bot[bot] avatar Jan 23 '25 15:01 eessi-bot[bot]

bot: build repo:eessi.io-2023.06-software arch:x86_64/amd/zen2

boegel avatar Jan 24 '25 11:01 boegel

Updates by the bot instance eessi-bot-mc-aws (click for details)
  • received bot command build repo:eessi.io-2023.06-software arch:x86_64/amd/zen2 from boegel

    • expanded format: build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen2
  • handling command build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen2 resulted in:

    • submitted job 42405, for details & status see https://github.com/EESSI/software-layer/pull/881#issuecomment-2612291182

eessi-bot[bot] avatar Jan 24 '25 11:01 eessi-bot[bot]

Updates by the bot instance eessi-bot-mc-azure (click for details)
  • received bot command build repo:eessi.io-2023.06-software arch:x86_64/amd/zen2 from boegel

    • expanded format: build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen2
  • handling command build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen2 resulted in:

    • no jobs were submitted

eessi-bot[bot] avatar Jan 24 '25 11:01 eessi-bot[bot]

New job on instance eessi-bot-mc-aws for CPU micro-architecture x86_64-amd-zen2 for repository eessi.io-2023.06-software in job dir /project/def-users/SHARED/jobs/2025.01/pr_881/42405

date job status comment
Jan 24 11:24:19 UTC 2025 submitted job id 42405 awaits release by job manager
Jan 24 11:24:53 UTC 2025 released job awaits launch by Slurm scheduler
Jan 24 11:25:55 UTC 2025 running job 42405 is running
Jan 24 11:33:02 UTC 2025 finished
:grin: SUCCESS (click triangle for details)
Details
:white_check_mark: job output file slurm-42405.out
:white_check_mark: no message matching FATAL:
:white_check_mark: no message matching ERROR:
:white_check_mark: no message matching FAILED:
:white_check_mark: no message matching required modules missing:
:white_check_mark: found message(s) matching No missing installations
:white_check_mark: found message matching .tar.gz created!
Artefacts
eessi-2023.06-software-linux-x86_64-amd-zen2-1737717953.tar.gzsize: 0 MiB (14895 bytes)
entries: 1
modules under 2023.06/software/linux/x86_64/amd/zen2/modules/all
no module files in tarball
software under 2023.06/software/linux/x86_64/amd/zen2/software
no software packages in tarball
other under 2023.06/software/linux/x86_64/amd/zen2
2023.06/init/easybuild/eb_hooks.py
Jan 24 11:33:02 UTC 2025 test result
:grin: SUCCESS (click triangle for details)
ReFrame Summary
[ OK ] ( 1/10) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/29Aug2024-foss-2023b-kokkos %scale=1_node /aeb2d9df @BotBuildTests:x86-64-amd-zen2-node+default
P: perf: 441.115 timesteps/s (r:0, l:None, u:None)
[ OK ] ( 2/10) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/2Aug2023_update2-foss-2023a-kokkos %scale=1_node /04ff9ece @BotBuildTests:x86-64-amd-zen2-node+default
P: perf: 427.9 timesteps/s (r:0, l:None, u:None)
[ OK ] ( 3/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node %device_type=cpu /775175bf @BotBuildTests:x86-64-amd-zen2-node+default
P: latency: 1.92 us (r:0, l:None, u:None)
[ OK ] ( 4/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node %device_type=cpu /52707c40 @BotBuildTests:x86-64-amd-zen2-node+default
P: latency: 1.77 us (r:0, l:None, u:None)
[ OK ] ( 5/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node %device_type=cpu /b1aacda9 @BotBuildTests:x86-64-amd-zen2-node+default
P: latency: 3.84 us (r:0, l:None, u:None)
[ OK ] ( 6/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node %device_type=cpu /c6bad193 @BotBuildTests:x86-64-amd-zen2-node+default
P: latency: 4.09 us (r:0, l:None, u:None)
[ OK ] ( 7/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node /15cad6c4 @BotBuildTests:x86-64-amd-zen2-node+default
P: latency: 0.57 us (r:0, l:None, u:None)
[ OK ] ( 8/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node /6672deda @BotBuildTests:x86-64-amd-zen2-node+default
P: latency: 0.58 us (r:0, l:None, u:None)
[ OK ] ( 9/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node /2a9a47b1 @BotBuildTests:x86-64-amd-zen2-node+default
P: bandwidth: 7444.26 MB/s (r:0, l:None, u:None)
[ OK ] (10/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node /1b24ab8e @BotBuildTests:x86-64-amd-zen2-node+default
P: bandwidth: 7388.1 MB/s (r:0, l:None, u:None)
[ PASSED ] Ran 10/10 test case(s) from 10 check(s) (0 failure(s), 0 skipped, 0 aborted)
Details
:white_check_mark: job output file slurm-42405.out
:white_check_mark: no message matching ERROR:
:white_check_mark: no message matching [\s*FAILED\s*].*Ran .* test case

eessi-bot[bot] avatar Jan 24 '25 11:01 eessi-bot[bot]

@bedroge can you move these changes to https://github.com/EESSI/software-layer-scripts

laraPPr avatar Jun 27 '25 14:06 laraPPr