easybuild-framework
easybuild-framework copied to clipboard
Remove output of module commands in log files
The output of the lmod command in the log makes the test report file miss (almost) all important info. E.g.: https://gist.github.com/Flamefire/704e041f5696e98b6b8ecb0ef98ec02f
I.e. there is a large chunk of those:
del os.environ["_ModuleTable071_"]
os.environ["_ModuleTable072_"] = ''
del os.environ["_ModuleTable072_"]
<etc>
...
_mlstatus = True
== 2025-06-25 17:00:38,652 run.py:611 INFO Warnings and errors of '/usr/share/lmod/lmod/libexec/lmod ...' shell command (stderr only):
== 2025-06-25 17:00:38,652 run.py:616 INFO Shell command completed successfully (see output above): /usr/share/lmod/lmod/libexec/lmod python unload TensorFlow/2.18.1-CUDA-12.6.0
== 2025-06-25 17:00:38,655 environment.py:93 INFO Environment variable LD_LIBRARY_PATH set to /software/protobuf/28.0-GCCcore-13.3.0/lib:/software/Abseil/20240722.0-GCCcore-13.3.0/lib:/software/snappy/1.2.1-GCCcore-13.3.0/lib:/software/libpng/1.6.43-GCCcore-13.3.0/lib:/software/nsync/1.29.2-GCCcore-13.3.0/lib:/software/libjpeg-turbo/3.0.1-GCCcore-13.3.0/lib:/software/JsonCpp/1.9.5-GCCcore-13.3.0/lib:/software/giflib/5.2.1-GCCcore-13.3.0/lib:/software/flatbuffers/24.3.25-GCCcore-13.3.0/lib:/software/double-conversion/3.3.0-GCCcore-13.3.0/lib:/software/HDF5/1.14.5-gompi-2024a/lib:/software/Szip/2.1.1-GCCcore-13.3.0/lib:/software/SciPy-bundle/2024.05-gfbf-2024a/lib/python3.12/site-packages/numpy/core/lib:/software/genoa/r24.04/NCCL/2.22.3-GCCcore-13.3.0-CUDA-12.6.0/lib:/software/genoa/r24.04/GDRCopy/2.4.1-GCCcore-13.3.0/lib:/software/cuDNN/9.5.0.50-CUDA-12.6.0/lib:/software/CUDA/12.6.0/nvvm/lib64:/software/CUDA/12.6.0/extras/CUPTI/lib64:/software/CUDA/12.6.0/targets/x86_64-linux/lib:/software/Boost/1.85.0-GCC-13.3.0/lib:/software/ICU/75.1-GCCcore-13.3.0/lib:/software/zstd/1.5.6-GCCcore-13.3.0/lib:/software/lz4/1.9.4-GCCcore-13.3.0/lib:/software/gettext/0.22.5-GCCcore-13.3.0/lib:/software/libiconv/1.17-GCCcore-13.3.0/lib:/software/genoa/r24.04/expat/2.6.2-GCCcore-13.3.0/lib:/software/genoa/r24.04/cURL/8.7.1-GCCcore-13.3.0/lib:/software/genoa/r24.04/Java/11.0.20/lib:/software/genoa/r24.04/Python/3.12.3-GCCcore-13.3.0/lib:/software/genoa/r24.04/libffi/3.4.5-GCCcore-13.3.0/lib64:/software/genoa/r24.04/SQLite/3.45.3-GCCcore-13.3.0/lib:/software/genoa/r24.04/Tcl/8.6.14-GCCcore-13.3.0/lib:/software/genoa/r24.04/libreadline/8.2-GCCcore-13.3.0/lib:/software/genoa/r24.04/ncurses/6.5-GCCcore-13.3.0/lib:/software/genoa/r24.04/bzip2/1.0.8-GCCcore-13.3.0/lib:/software/genoa/r24.04/ScaLAPACK/2.2.0-gompi-2024a-fb/lib:/software/genoa/r24.04/FFTW.MPI/3.3.10-gompi-2024a/lib:/software/genoa/r24.04/FFTW/3.3.10-GCC-13.3.0/lib:/software/genoa/r24.04/FlexiBLAS/3.4.4-GCC-13.3.0/lib:/software/genoa/r24.04/OpenBLAS/0.3.27-GCC-13.3.0/lib:/software/genoa/r24.04/OpenMPI/5.0.3-GCC-13.3.0/lib:/software/genoa/r24.04/UCC/1.3.0-GCCcore-13.3.0/lib:/software/genoa/r24.04/PRRTE/3.0.5-GCCcore-13.3.0/lib:/software/genoa/r24.04/PMIx/5.0.2-GCCcore-13.3.0/lib:/software/genoa/r24.04/libfabric/1.21.0-GCCcore-13.3.0/lib:/software/genoa/r24.04/UCX/1.16.0-GCCcore-13.3.0/lib:/software/genoa/r24.04/libevent/2.1.12-GCCcore-13.3.0/lib:/software/genoa/r24.04/OpenSSL/3/lib:/software/genoa/r24.04/hwloc/2.10.0-GCCcore-13.3.0/lib:/software/genoa/r24.04/libpciaccess/0.18.1-GCCcore-13.3.0/lib:/software/genoa/r24.04/libxml2/2.12.7-GCCcore-13.3.0/lib:/software/genoa/r24.04/XZ/5.4.5-GCCcore-13.3.0/lib:/software/genoa/r24.04/numactl/2.0.18-GCCcore-13.3.0/lib:/software/binutils/2.42-GCCcore-13.3.0/lib:/software/zlib/1.3.1-GCCcore-13.3.0/lib:/software/genoa/r24.04/GCCcore/13.3.0/lib64 (previously undefined)
== 2025-06-25 17:00:38,657 filetools.py:403 INFO Path /tmp/easybuild-tmp/eb-gehzfysz/tmp7vzo8jyc successfully removed.
== 2025-06-25 17:00:38,657 environment.py:93 INFO Environment variable _ModuleTable002_ set to IjAyMDI0MDcyMi4qemZpbmFsIiwKfSwKQmF6ZWwgPSB7CmZuID0gIi9kYXRhL2hvcnNlL3dzL3MzMjQ4OTczLUVhc3lCdWlsZC9lYXN5YnVpbGQtZ2Vub2EvbW9kdWxlcy9hbGwvQ29tcGlsZXIvR0NDY29yZS8xMy4zLjAvQmF6ZWwvNi41LjAtSmF2YS0xMS5sdWEiLApmdWxsTmFtZSA9ICJCYXplbC82LjUuMC1KYXZhLTExIiwKbG9hZE9yZGVyID0gMjcsCnByb3BUID0ge30sCnN0YWNrRGVwdGggPSAwLApzdGF0dXMgPSAiYWN0aXZlIiwKdXNlck5hbWUgPSAiQmF6ZWwvNi41LjAtSmF2YS0xMSIsCndWID0gIjAwMDAwMDAwNi4wMDAwMDAwMDUuKmphdmEuKnpmaW5hbC0uMDAwMDAwMDExLip6ZmluYWwiLAp9LApCb29zdCA9IHsKZm4gPSAiL2RhdGEvaG9yc2Uvd3MvczMyNDg5 (previous value: 'LjAwMDAwMDAwNS4qamF2YS4qemZpbmFsLS4wMDAwMDAwMTEuKnpmaW5hbCIsCn0sCkJvb3N0ID0gewpmbiA9ICIvZGF0YS9ob3JzZS93cy9zMzI0ODk3My1FYXN5QnVpbGQvZWFzeWJ1aWxkLWdlbm9hL21vZHVsZXMvYWxsL0NvbXBpbGVyL0dDQy8xMy4zLjAvQm9vc3QvMS44NS4wLmx1YSIsCmZ1bGxOYW1lID0gIkJvb3N0LzEuODUuMCIsCmxvYWRPcmRlciA9IDQ2LApwcm9wVCA9IHt9LApyZWZfY291bnQgPSAxLApzdGFja0RlcHRoID0gMSwKc3RhdHVzID0gImFjdGl2ZSIsCnVzZXJOYW1lID0gIkJvb3N0LzEuODUuMCIsCndWID0gIjAwMDAwMDAwMS4wMDAwMDAwODUuKnpmaW5hbCIsCn0sCkN5dGhvbiA9IHsKZm4gPSAiL2RhdGEvaG9yc2Uvd3MvczMyNDg5NzMtRWFzeUJ1')
== 2025-06-25 17:00:38,657 environment.py:93 INFO Environment variable __LMOD_REF_COUNT_MODULEPATH set to /modules/all:1;/modules/all/MPI/GCC/13.3.0/OpenMPI/5.0.3:1;/software/modules/genoa/r24.04/all/MPI/GCC/13.3.0/OpenMPI/5.0.3:1;/modules/all/Compiler/GCC/13.3.0:1;/software/modules/genoa/r24.04/all/Compiler/GCC/13.3.0:1;/modules/all/Compiler/GCCcore/13.3.0:1;/software/modules/genoa/r24.04/all/Compiler/GCCcore/13.3.0:1;/modules/all/Core:1;/home/s3248973/.local/EasyBuildDev/modules:1;/software/modules/genoa/r24.04/all:1;/software/modules/genoa/r24.04/all/Core:1;/software/modules/releases/genoa:1 (previous value: '/modules/all/MPI/GCC/13.3.0/OpenMPI/5.0.3:2;/software/modules/genoa/r24.04/all/MPI/GCC/13.3.0/OpenMPI/5.0.3:2;/modules/all/Compiler/GCC/13.3.0:2;/software/modules/genoa/r24.04/all/Compiler/GCC/13.3.0:2;/modules/all/Compiler/GCCcore/13.3.0:2;/software/modules/genoa/r24.04/all/Compiler/GCCcore/13.3.0:2;/modules/all/Core:1;/tmp/easybuild-tmp/eb-gehzfysz/tmp7vzo8jyc/all/MPI/GCC/13.3.0/OpenMPI/5.0.3:1;/modules/all:1;/home/s3248973/.local/EasyBuildDev/modules:1;/software/modules/genoa/r24.04/all:1;/software/modules/genoa/r24.04/all/Core:1;/software/modules/releases/genoa:1')
<etc>
...
Do we really need the output of the module commands or can we silence them completely?