easybuild-easyblocks icon indicating copy to clipboard operation
easybuild-easyblocks copied to clipboard

Published 20 hours ago • verified publisher icon easybuilders

Enhance generic Bundle EasyBlock to transfer module requirements of components

Open Thyre opened this issue 1 year ago • 12 comments

This PR implements the make_module_req_guess method for the generic Bundle EasyBlock. With this, all the requirements of the components in a bundle are transferred correctly to the final module. Previously, this could lead to missing environment variables, letting the build succeed but still resulting in a broken module, for example because PATH is not set.

Fixes https://github.com/easybuilders/easybuild-easyblocks/issues/2733

Thyre avatar Oct 07 '24 16:10 Thyre

I've only tested the changes with https://github.com/easybuilders/easybuild-easyconfigs/pull/21582 so far. I'll try to gather the EasyConfigs using Bundle in some way to make sure that I did not break them, since this is a change on a widely used EasyBlock.

Unfortunately, I can only test GCC 12.3.0 and newer easily and are somewhat limited with my compute resources. So this might take a while..

Thyre avatar Oct 07 '24 16:10 Thyre

Example with the changed EasyBlock:

https://github.com/easybuilders/easybuild-easyconfigs/pull/21582#issuecomment-2397359586

Thyre avatar Oct 07 '24 16:10 Thyre

Test report by @Thyre

Overview of tested easyconfigs (in order)

  • SUCCESS FlexiBLAS-3.4.4-GCC-13.3.0.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total) datenlager - Linux Ubuntu 24.04, x86_64, AMD Ryzen 7 3700X 8-Core Processor, Python 3.12.3 See https://gist.github.com/Thyre/a33ae670ab1932bdcfff62bae82c3f53 for a full test report.

Thyre avatar Oct 07 '24 16:10 Thyre

Test report by @Thyre

Overview of tested easyconfigs (in order)

  • SUCCESS GCC-13.3.0.eb
  • SUCCESS Autotools-20231222-GCCcore-13.3.0.eb
  • SUCCESS cffi-1.16.0-GCCcore-13.3.0.eb
  • SUCCESS Julia-1.9.2-linux-x86_64.eb
  • SUCCESS groff-1.23.0-GCCcore-13.3.0.eb
  • SUCCESS Circuitscape-5.12.3-Julia-1.9.2.eb
  • SUCCESS expat-2.6.2-GCCcore-13.3.0.eb
  • SUCCESS libiconv-1.17-GCCcore-13.3.0.eb
  • SUCCESS patchelf-0.18.0-GCCcore-13.3.0.eb
  • SUCCESS Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb
  • SUCCESS gettext-0.22.5-GCCcore-13.3.0.eb
  • SUCCESS Rust-1.78.0-GCCcore-13.3.0.eb
  • SUCCESS intltool-0.51.0-GCCcore-13.3.0.eb
  • SUCCESS Doxygen-1.11.0-GCCcore-13.3.0.eb
  • SUCCESS hatchling-1.24.2-GCCcore-13.3.0.eb
  • SUCCESS setuptools-rust-1.9.0-GCCcore-13.3.0.eb
  • SUCCESS maturin-1.6.0-GCCcore-13.3.0.eb
  • SUCCESS typing-extensions-4.11.0-GCCcore-13.3.0.eb
  • FAIL (build issue) pydantic-2.9.1-GCCcore-13.3.0.eb (partial log available at https://gist.github.com/Thyre/abd4413683d3801209f2daa0a6d77ed8)
  • SUCCESS gperf-3.1-GCCcore-13.3.0.eb
  • SUCCESS util-linux-2.40-GCCcore-13.3.0.eb
  • SUCCESS libpng-1.6.43-GCCcore-13.3.0.eb
  • SUCCESS Brotli-1.1.0-GCCcore-13.3.0.eb
  • SUCCESS freetype-2.13.2-GCCcore-13.3.0.eb
  • SUCCESS fontconfig-2.15.0-GCCcore-13.3.0.eb
  • SUCCESS X11-20240607-GCCcore-13.3.0.eb

Build succeeded for 25 out of 26 (7 easyconfigs in total) datenlager - Linux Ubuntu 24.04, x86_64, AMD Ryzen 7 3700X 8-Core Processor, Python 3.12.3 See https://gist.github.com/Thyre/3b8848b109d905611283a882db8bb378 for a full test report.

The failed one doesn't look related:

Building a mixed python/rust project
       Found pyo3 bindings
       maturin failed
        Caused by: Python interpreter should be a kind of interpreter (e.g. 'python3.8' or 'pypy3.9') when cross-compiling, got path to interpreter: /tank/Programs/Linux/EasyBuild/datenlager/Software/software/Python/3.12.3-GCCcore-13.3.0/bin/python
      Error running maturin: Command '['maturin', 'pep517', 'write-dist-info', '--metadata-directory', '/tmp/eb-1tc417di/pip-modern-metadata-s3zbqxe6', '--interpreter', '/tank/Programs/Linux/EasyBuild/datenlager/Software/software/Python/3.12.3-GCCcore-13.3.0/bin/python']' returned non-zero exit status 1.
      Checking for Rust toolchain....
      Running `maturin pep517 write-dist-info --metadata-directory /tmp/eb-1tc417di/pip-modern-metadata-s3zbqxe6 --interpreter /tank/Programs/Linux/EasyBuild/datenlager/Software/software/Python/3.12.3-GCCcore-13.3.0/bin/python`
      [end of output]

The tested ones include normal Bundle (e.g. GCC & Autotools), PerlBundle (Perl-bundle-CPAN), PythonBundle (cffi) and JuliaBundle (Circuitscape). CargoPythonBundle failed due to the failure shown above.

Thyre avatar Oct 07 '24 18:10 Thyre

Changed PR so that the EasyBlock for each component is stored and not initialized in multiple steps.

Thyre avatar Oct 11 '24 06:10 Thyre

Based on https://docs.easybuild.io/version-specific/easyblocks/?h=bundle, this change should affect the following EasyBlocks:

  • Bundle
  • BuildEnv
  • CrayToolchain
  • EB_Clang_minus_AOMP
  • EB_OpenSSL_wrapper
  • JuliaBundle
  • PerlBundle
  • PythonBundle
  • CargoPythonBundle
  • SystemCompiler
  • SystemMPI
  • Toolchain

I will not be able to test all of them (e.g. CrayToolchain), but I'll try to get one EasyConfig for all of these EasyBlocks to make sure that nothing broke and compare their modules.

Thyre avatar Oct 11 '24 06:10 Thyre

I've chosen a few EasyConfigs to compare with this MR and the current develop.

Building will take quite some time. I'll post the resulting module diffs later.

Thyre avatar Oct 25 '24 09:10 Thyre

Now that all modules are finally installed (but I had to partially select other modules due to general installation issues), I found one bug with the proposed changes:

When checking crypt4gh-1.7-GCC-12.3.0.eb, the module does a sanity check for an executable in /bin. This path is correctly added in develop:

help([==[

Description
===========
crypt4gh is a Python tool to encrypt, decrypt or re-encrypt files, 
according to the GA4GH encryption file format.


More information
================
 - Homepage: https://github.com/EGA-archive/crypt4gh


Included extensions
===================
crypt4gh-1.7
]==])

whatis([==[Description: crypt4gh is a Python tool to encrypt, decrypt or re-encrypt files, 
according to the GA4GH encryption file format.]==])
whatis([==[Homepage: https://github.com/EGA-archive/crypt4gh]==])
whatis([==[URL: https://github.com/EGA-archive/crypt4gh]==])
whatis([==[Extensions: crypt4gh-1.7]==])

local root = "/tank/Programs/Linux/EasyBuild/datenlager/Software/software/crypt4gh/1.7-GCC-12.3.0"

conflict("crypt4gh")

if not ( isloaded("GCC/12.3.0") ) then
    load("GCC/12.3.0")
end

if not ( isloaded("Python/3.11.3-GCCcore-12.3.0") ) then
    load("Python/3.11.3-GCCcore-12.3.0")
end

if not ( isloaded("Python-bundle-PyPI/2023.06-GCCcore-12.3.0") ) then
    load("Python-bundle-PyPI/2023.06-GCCcore-12.3.0")
end

if not ( isloaded("PyYAML/6.0-GCCcore-12.3.0") ) then
    load("PyYAML/6.0-GCCcore-12.3.0")
end

if not ( isloaded("cryptography/41.0.1-GCCcore-12.3.0") ) then
    load("cryptography/41.0.1-GCCcore-12.3.0")
end

if not ( isloaded("bcrypt/4.0.1-GCCcore-12.3.0") ) then
    load("bcrypt/4.0.1-GCCcore-12.3.0")
end

prepend_path("CMAKE_PREFIX_PATH", root)
prepend_path("LIBRARY_PATH", pathJoin(root, "lib"))
prepend_path("PATH", pathJoin(root, "bin"))
setenv("EBROOTCRYPT4GH", root)
setenv("EBVERSIONCRYPT4GH", "1.7")
setenv("EBDEVELCRYPT4GH", pathJoin(root, "easybuild/crypt4gh-1.7-GCC-12.3.0-easybuild-devel"))

prepend_path("PYTHONPATH", pathJoin(root, "lib/python3.11/site-packages"))
-- Built with EasyBuild version 4.9.5.dev0
setenv("EBEXTSLISTCRYPT4GH", "crypt4gh-1.7")

but this fails with the proposed changes. CMAKE_PREFIX_PATH, LIBRARY_PATH and PATH are missing. I suspect that adding make_module_req_guess to Bundle causes this to fail.

Thyre avatar Oct 26 '24 16:10 Thyre

Test report by @Thyre

Overview of tested easyconfigs (in order)

  • SUCCESS buildenv-default-foss-2023b.eb
  • SUCCESS OpenSSL-3.eb
  • SUCCESS Circuitscape-5.12.3-Julia-1.9.2.eb
  • SUCCESS Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb
  • SUCCESS gfbf-2024a.eb
  • SUCCESS crypt4gh-1.7-GCC-12.3.0.eb
  • SUCCESS GCC-system.eb
  • SUCCESS OpenMPI-system-GCC-system-2.29.eb
  • SUCCESS X11-20240607-GCCcore-13.3.0.eb

Build succeeded for 9 out of 9 (9 easyconfigs in total) datenlager - Linux Ubuntu 24.04, x86_64, AMD Ryzen 7 3700X 8-Core Processor, Python 3.12.3 See https://gist.github.com/Thyre/75aa472a40750fd854cf2d3ec2ede6e3 for a full test report.

I'm not able to test jiter-0.4.1-GCCcore-12.3.0.eb due to the following error caused by maturin:

  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error

   Preparing metadata (pyproject.toml) did not run successfully.
   exit code: 1
  > [6 lines of output]
       Found pyo3 bindings
       maturin failed
        Caused by: Python interpreter should be a kind of interpreter (e.g. 'python3.8' or 'pypy3.9') when cross-compiling, got path to interpreter: /tank/Programs/Linux/EasyBuild/datenlager/Software/software/Python/3.11.3-GCCcore-12.3.0/bin/python
      Error running maturin: Command '['maturin', 'pep517', 'write-dist-info', '--metadata-directory', '/tmp/eb-ibucduig/pip-modern-metadata-debc3bzs', '--interpreter', '/tank/Programs/Linux/EasyBuild/datenlager/Software/software/Python/3.11.3-GCCcore-12.3.0/bin/python']' returned non-zero exit status 1.
      Checking for Rust toolchain....
      Running `maturin pep517 write-dist-info --metadata-directory /tmp/eb-ibucduig/pip-modern-metadata-debc3bzs --interpreter /tank/Programs/Linux/EasyBuild/datenlager/Software/software/Python/3.11.3-GCCcore-12.3.0/bin/python`
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

 Encountered error while generating package metadata.
> See above for output.

I've also checked if the module files of the EasyConfigs above changed. They did not.

Thyre avatar Oct 26 '24 18:10 Thyre

Full (re-)installation

Test report by @Thyre

Overview of tested easyconfigs (in order)

  • SUCCESS buildenv-default-foss-2023b.eb
  • SUCCESS OpenSSL-3.eb
  • SUCCESS Circuitscape-5.12.3-Julia-1.9.2.eb
  • SUCCESS Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb
  • SUCCESS gfbf-2024a.eb
  • SUCCESS crypt4gh-1.7-GCC-12.3.0.eb
  • SUCCESS GCC-system.eb
  • SUCCESS OpenMPI-system-GCC-system-2.29.eb
  • SUCCESS X11-20240607-GCCcore-13.3.0.eb

Build succeeded for 9 out of 9 (9 easyconfigs in total) datenlager - Linux Ubuntu 24.04, x86_64, AMD Ryzen 7 3700X 8-Core Processor, Python 3.12.3 See https://gist.github.com/Thyre/bbc3468bc740b0c511c9f1e665d32910 for a full test report.

Thyre avatar Oct 26 '24 20:10 Thyre

Module only

Test report by @Thyre

Overview of tested easyconfigs (in order)

  • SUCCESS buildenv-default-foss-2023b.eb
  • SUCCESS OpenSSL-3.eb
  • SUCCESS Circuitscape-5.12.3-Julia-1.9.2.eb
  • SUCCESS Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb
  • SUCCESS gfbf-2024a.eb
  • SUCCESS crypt4gh-1.7-GCC-12.3.0.eb
  • SUCCESS GCC-system.eb
  • SUCCESS OpenMPI-system-GCC-system-2.29.eb
  • SUCCESS X11-20240607-GCCcore-13.3.0.eb

Build succeeded for 9 out of 9 (9 easyconfigs in total) datenlager - Linux Ubuntu 24.04, x86_64, AMD Ryzen 7 3700X 8-Core Processor, Python 3.12.3 See https://gist.github.com/Thyre/c78da935cdd54f14102e4d932295777e for a full test report.

Thyre avatar Oct 26 '24 21:10 Thyre

Testing a few of the Stages 2025 EasyConfigs using Bundle/PythonBundle/... of the JSC repo

Test report by @Thyre

Overview of tested easyconfigs (in order)

  • SUCCESS Autotools-20231222-GCCcore-13.3.0.eb
  • SUCCESS OpenSSL-3.eb
  • SUCCESS Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb
  • SUCCESS boto3-1.35.36-GCCcore-13.3.0.eb
  • SUCCESS npsmpic-2024a.eb

Build succeeded for 5 out of 5 (5 easyconfigs in total) jrlogin05.jureca - Linux Rocky Linux 9.4, x86_64, AMD EPYC 7742 64-Core Processor, 2 x NVIDIA Quadro RTX 8000, 560.35.03, Python 3.9.18 See https://gist.github.com/Thyre/81e26fbe93c25adabffd33e5664ddc0b for a full test report.

Thyre avatar Oct 27 '24 17:10 Thyre

Bi-weekly EasyBuild conf call 2024-11-06: While these changes continue to work with EasyBuild 5.0.x, the make_module_req_guess method will be deprecated.

My current idea would be to continue with this PR as it is. Once https://github.com/easybuilders/easybuild-framework/pull/4653 is merged, I'll work on porting these changes to the new method for EasyBuild 5.0.x.

Thyre avatar Nov 06 '24 10:11 Thyre

@boegelbot please test @ jsc-zen3 EB_ARGS="X11-20240607-GCCcore-13.3.0.eb buildenv-default-foss-2023b.eb OpenSSL-3.eb Circuitscape-5.12.3-Julia-1.9.2.eb Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb crypt4gh-1.7-GCC-12.3.0.eb jiter-0.4.1-GCCcore-12.3.0.eb GCC-system.eb gfbf-2024a.eb" CORE_CNT=16

Thyre avatar Nov 08 '24 09:11 Thyre

@Thyre: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3472 EB_ARGS="X11-20240607-GCCcore-13.3.0.eb buildenv-default-foss-2023b.eb OpenSSL-3.eb Circuitscape-5.12.3-Julia-1.9.2.eb Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb crypt4gh-1.7-GCC-12.3.0.eb jiter-0.4.1-GCCcore-12.3.0.eb GCC-system.eb gfbf-2024a.eb" EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3472 --ntasks="16" ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 5224

Test results coming soon (I hope)...

- notification for comment with ID 2464270943 processed

Message to humans: this is just bookkeeping information for me, it is of no use to you (unless you think I have a bug, which I don't).

boegelbot avatar Nov 08 '24 09:11 boegelbot

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • SUCCESS X11-20240607-GCCcore-13.3.0.eb
  • SUCCESS buildenv-default-foss-2023b.eb
  • SUCCESS OpenSSL-3.eb
  • SUCCESS Circuitscape-5.12.3-Julia-1.9.2.eb
  • SUCCESS Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb
  • SUCCESS crypt4gh-1.7-GCC-12.3.0.eb
  • SUCCESS jiter-0.4.1-GCCcore-12.3.0.eb
  • SUCCESS GCC-system.eb
  • SUCCESS gfbf-2024a.eb

Build succeeded for 9 out of 9 (9 easyconfigs in total) jsczen3c2.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.4, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18 See https://gist.github.com/boegelbot/f0ee3fa8737148ccb5fae372b743d256 for a full test report.

boegelbot avatar Nov 08 '24 11:11 boegelbot

@boegelbot please test @ jsc-zen3 EB_ARGS="--module-only X11-20240607-GCCcore-13.3.0.eb buildenv-default-foss-2023b.eb OpenSSL-3.eb Circuitscape-5.12.3-Julia-1.9.2.eb Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb crypt4gh-1.7-GCC-12.3.0.eb jiter-0.4.1-GCCcore-12.3.0.eb GCC-system.eb gfbf-2024a.eb"

Thyre avatar Nov 08 '24 11:11 Thyre

@Thyre: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3472 EB_ARGS="--module-only X11-20240607-GCCcore-13.3.0.eb buildenv-default-foss-2023b.eb OpenSSL-3.eb Circuitscape-5.12.3-Julia-1.9.2.eb Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb crypt4gh-1.7-GCC-12.3.0.eb jiter-0.4.1-GCCcore-12.3.0.eb GCC-system.eb gfbf-2024a.eb" EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3472 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 5226

Test results coming soon (I hope)...

- notification for comment with ID 2464420612 processed

Message to humans: this is just bookkeeping information for me, it is of no use to you (unless you think I have a bug, which I don't).

boegelbot avatar Nov 08 '24 11:11 boegelbot

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • SUCCESS X11-20240607-GCCcore-13.3.0.eb
  • SUCCESS buildenv-default-foss-2023b.eb
  • SUCCESS OpenSSL-3.eb
  • SUCCESS Circuitscape-5.12.3-Julia-1.9.2.eb
  • SUCCESS Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb
  • SUCCESS crypt4gh-1.7-GCC-12.3.0.eb
  • SUCCESS jiter-0.4.1-GCCcore-12.3.0.eb
  • SUCCESS GCC-system.eb
  • SUCCESS gfbf-2024a.eb

Build succeeded for 9 out of 9 (9 easyconfigs in total) jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.4, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18 See https://gist.github.com/boegelbot/b951173c327e4bb0f8b38422bdf91456 for a full test report.

boegelbot avatar Nov 08 '24 11:11 boegelbot

Thanks a lot for the suggestions. I'll add them to the commit in a moment 😄

Thyre avatar Nov 08 '24 12:11 Thyre

@boegelbot please test @ jsc-zen3 EB_ARGS="--module-only X11-20240607-GCCcore-13.3.0.eb buildenv-default-foss-2023b.eb OpenSSL-3.eb Circuitscape-5.12.3-Julia-1.9.2.eb Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb crypt4gh-1.7-GCC-12.3.0.eb jiter-0.4.1-GCCcore-12.3.0.eb GCC-system.eb gfbf-2024a.eb"

Thyre avatar Nov 08 '24 14:11 Thyre

@Thyre: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3472 EB_ARGS="--module-only X11-20240607-GCCcore-13.3.0.eb buildenv-default-foss-2023b.eb OpenSSL-3.eb Circuitscape-5.12.3-Julia-1.9.2.eb Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb crypt4gh-1.7-GCC-12.3.0.eb jiter-0.4.1-GCCcore-12.3.0.eb GCC-system.eb gfbf-2024a.eb" EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3472 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 5235

Test results coming soon (I hope)...

- notification for comment with ID 2464855921 processed

Message to humans: this is just bookkeeping information for me, it is of no use to you (unless you think I have a bug, which I don't).

boegelbot avatar Nov 08 '24 14:11 boegelbot

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • SUCCESS X11-20240607-GCCcore-13.3.0.eb
  • SUCCESS buildenv-default-foss-2023b.eb
  • SUCCESS OpenSSL-3.eb
  • SUCCESS Circuitscape-5.12.3-Julia-1.9.2.eb
  • SUCCESS Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb
  • SUCCESS crypt4gh-1.7-GCC-12.3.0.eb
  • SUCCESS jiter-0.4.1-GCCcore-12.3.0.eb
  • SUCCESS GCC-system.eb
  • SUCCESS gfbf-2024a.eb

Build succeeded for 9 out of 9 (9 easyconfigs in total) jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.4, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18 See https://gist.github.com/boegelbot/2df274b1d2731957ac6b0e5be8d36daf for a full test report.

boegelbot avatar Nov 08 '24 14:11 boegelbot

@Thyre thanks for the quick update! do you want to add your name to the list of contributors to this easyblock? If you do, there is no need to send new tests again :slightly_smiling_face:

lexming avatar Nov 08 '24 14:11 lexming

@Thyre thanks for the quick update! do you want to add your name to the list of contributors to this easyblock?

Done. Took longer than I want to admit to find the authors comment 😅

If you do, there is no need to send new tests again 🙂

Agreed, changing the comment shouldn't break anything. Just wanted to make sure nothing broke during the last changes.

Thyre avatar Nov 08 '24 14:11 Thyre

Merging, thanks @Thyre !

lexming avatar Nov 09 '24 18:11 lexming

@Thyre @lexming I think the changes here may lead to an infinite loop, the easyconfigs test suite is failing due to generating too many open files, and it seems to coincide with this PR getting merged...

boegel avatar Nov 12 '24 17:11 boegel

@Thyre @lexming I think the changes here may lead to an infinite loop, the easyconfigs test suite is failing due to generating too many open files, and it seems to coincide with this PR getting merged...

I've opened #3504 to revert the changes that were done here. Purely based on timing, it seems like the changes here are the culprit for the trouble we're currently seeing in the easyconfigs test suite...

boegel avatar Nov 12 '24 17:11 boegel

@Thyre the changes in this PR are still alive in 5.0.x branch (https://github.com/easybuilders/easybuild-easyblocks/blob/5.0.x/easybuild/easyblocks/generic/bundle.py). We can focus on that branch for trobleshooting and fixing.

lexming avatar Nov 12 '24 22:11 lexming

@Thyre the changes in this PR are still alive in 5.0.x branch (https://github.com/easybuilders/easybuild-easyblocks/blob/5.0.x/easybuild/easyblocks/generic/bundle.py). We can focus on that branch for trobleshooting and fixing.

@lexming We found the issue, see https://github.com/easybuilders/easybuild-easyconfigs/issues/21841. It basically just boils down to loggers not being closed during the EasyConfig tests. I'll prepare an updated PR with a fix for this (and the comment regarding clang_aomp.py) tomorrow.

Thyre avatar Nov 12 '24 22:11 Thyre