easybuild-easyblocks icon indicating copy to clipboard operation
easybuild-easyblocks copied to clipboard

Published 20 hours ago • verified publisher icon easybuilders

update `ConfigureMake` easyblock to error out on unknown configure args

Open Flamefire opened this issue 2 years ago • 11 comments

(created using eb --new-pr)

Fixes #157

This may cause failures in some existing EasyConfigs but actually we shouldn't have any unknown args in use and if we did that easyconfig likely needs fixing as it may not do what it is supposed to or looks like it is doing.

Requires:

  • https://github.com/easybuilders/easybuild-easyblocks/pull/3087

As identified in #3026 this should be tested with:

  • [x] BigDFT-1.9.1-foss-2021b.eb (Binary EasyBlock: https://github.com/easybuilders/easybuild-easyblocks/issues/3252)
  • [x] bigdft-suite-1.9.1-foss-2021b.eb (This is actually the same as above)
  • [x] FFLAS-FFPACK-2.5.0-gfbf-2022a.eb https://github.com/easybuilders/easybuild-easyconfigs/pull/20052
  • [x] FriBidi: unrecognized options: --disable-docs, fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/19147 + https://github.com/easybuilders/easybuild-easyconfigs/pull/19998
    • [x] FriBidi-1.0.10-GCCcore-10.2.0.eb
    • [x] FriBidi-1.0.10-GCCcore-10.3.0.eb
    • [x] FriBidi-1.0.10-GCCcore-11.2.0.eb
    • [x] FriBidi-1.0.12-GCCcore-11.3.0.eb
    • [x] FriBidi-1.0.12-GCCcore-12.2.0.eb
    • [x] FriBidi-1.0.12-GCCcore-12.3.0.eb
    • [x] FriBidi-1.0.13-GCCcore-13.2.0.eb
    • [x] FriBidi-1.0.5-GCCcore-7.3.0.eb
    • [x] FriBidi-1.0.5-GCCcore-8.2.0.eb
    • [x] FriBidi-1.0.5-GCCcore-8.3.0.eb
    • [x] FriBidi-1.0.9-GCCcore-9.3.0.eb
  • [ ] gap-4.12.2-foss-2022a.eb
  • [x] GDAL: unrecognized options: --with-libgeotiff, fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/19999
    • [x] GDAL-2.2.3-foss-2018b-Python-2.7.15.eb
    • [x] GDAL-2.2.3-foss-2018b-Python-3.6.6.eb
    • [x] GDAL-3.0.0-foss-2019a-Python-3.7.2.eb
    • [x] GDAL-3.0.2-foss-2019b-Python-3.7.4.eb
    • [x] GDAL-3.0.2-intel-2019b-Python-3.7.4.eb
    • [x] GDAL-3.0.4-foss-2020a-Python-3.8.2.eb
  • [x] GDAL-3.5.0-foss-2022a.eb: unrecognized options: --with-jasper, fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/20009
  • [ ] git-annex-10.20230802-GCCcore-12.2.0.eb
  • [x] GnuTLS-3.7.2-GCCcore-10.3.0.eb (fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/20065)
  • [x] GRASS-8.2.0-foss-2021b.eb (fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/20062)
  • [x] groff-1.23.0-GCCcore-13.2.0.eb (fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/19969)
  • [x] GTK+, GTK3: unrecognized options: --disable-visibility, fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/20028
    • [x] GTK+-3.24.13-GCCcore-8.3.0.eb
    • [x] GTK+-3.24.17-GCCcore-9.3.0.eb
    • [x] GTK+-3.24.23-GCCcore-10.2.0.eb
    • [x] GTK+-3.24.8-GCCcore-8.2.0.eb
    • [x] GTK3-3.24.29-GCCcore-10.3.0.eb
    • [x] GTK3-3.24.31-GCCcore-11.2.0.eb
    • [x] GTK3-3.24.33-GCCcore-11.3.0.eb
    • [x] GTK3-3.24.35-GCCcore-12.2.0.eb
  • [x] HDF-EOS2-3.0-GCCcore-11.3.0.eb (fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/20033)
  • [x] hwloc: unrecognized options: --enable-libnuma, fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/19833 + https://github.com/easybuilders/easybuild-easyconfigs/pull/19085
    • [x] hwloc-2.0.2-GCCcore-8.2.0.eb
    • [x] hwloc-2.0.3-GCCcore-8.3.0.eb
    • [x] hwloc-2.1.0-GCCcore-9.2.0.eb
    • [x] hwloc-2.2.0-GCCcore-10.2.0.eb
    • [x] hwloc-2.2.0-GCCcore-9.3.0.eb
    • [x] hwloc-2.4.1-GCCcore-10.3.0.eb
    • [x] hwloc-2.5.0-GCCcore-11.2.0.eb
    • [x] hwloc-2.7.1-GCCcore-11.3.0.eb
    • [x] hwloc-2.8.0-GCCcore-12.2.0.eb
    • [x] hwloc-2.9.1-GCCcore-12.3.0.eb
    • [x] hwloc-2.9.2-GCCcore-13.2.0.eb
  • libfdf: unrecognized options: --without-mpi, fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/20034
    • [x] libfdf-0.2.2-GCC-10.2.0-serial.eb
    • [x] libfdf-0.2.2-GCC-10.3.0-serial.eb
    • [x] libfdf-0.2.2-GCC-11.2.0-serial.eb
    • [x] libfdf-0.2.2-iccifort-2020.4.304-serial.eb
    • [x] libfdf-0.2.2-intel-compilers-2021.2.0-serial.eb
    • [x] libfdf-0.2.2-intel-compilers-2021.4.0-serial.eb
  • [x] Libint: unrecognized options: --with-cxxgen-optflags, fixed in #3249
    • [x] Libint-2.6.0-GCC-10.2.0-lmax-6-cp2k.eb
    • [x] Libint-2.6.0-GCC-10.3.0-lmax-6-cp2k.eb
    • [x] Libint-2.6.0-GCC-11.3.0-lmax-6-cp2k.eb
    • [x] Libint-2.6.0-gompi-2020a-lmax-6-cp2k.eb
    • [x] Libint-2.6.0-iccifort-2020.4.304-lmax-6-cp2k.eb
    • [x] Libint-2.6.0-iimpi-2021a-lmax-6-cp2k.eb
  • [x] LibTIFF-4.1.0-GCCcore-10.2.0.eb (fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/19951)
  • [x] LibTIFF-4.1.0-GCCcore-8.3.0.eb (fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/19951)
  • [x] LibTIFF-4.1.0-GCCcore-9.3.0.eb (fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/19951)
  • [x] LinBox-1.7.0-gfbf-2022a.eb (fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/20064)
  • [x] MDSplus-7.96.12-GCCcore-9.3.0.eb (fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/20041)
  • [x] MEME-5.4.1-GCC-10.3.0.eb (fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/20043)
  • [x] MEME-5.4.1-GCC-11.2.0.eb (fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/20043)
  • [ ] MESS-0.1.6-foss-2019b.eb
  • [x] MIRA-5.0rc2-foss-2020b.eb (fixed in https://github.com/easybuilders/easybuild-easyconfigs/pull/20061)
  • [x] OpenMPI: unrecognized options: --with-ucc, fixed in #3223
    • [x] OpenMPI-4.0.6-GCC-10.3.0.eb
    • [x] OpenMPI-4.0.7-GCC-10.3.0.eb
    • [x] OpenMPI-4.1.1-GCC-10.3.0.eb
    • [x] OpenMPI-4.1.1-GCC-11.2.0.eb
    • [x] OpenMPI-4.1.1-intel-compilers-2021.4.0.eb
    • [x] OpenMPI-4.1.2-GCC-11.2.0.eb
  • [x] Tcl => only internal unknown configure options, during build step, so unaffected
    • Tcl-8.6.10-GCCcore-10.2.0.eb
    • Tcl-8.6.10-GCCcore-9.3.0.eb
    • Tcl-8.6.11-GCCcore-10.3.0.eb
    • Tcl-8.6.11-GCCcore-11.2.0.eb
    • Tcl-8.6.12-GCCcore-11.3.0.eb
    • Tcl-8.6.12-GCCcore-12.2.0.eb
    • Tcl-8.6.13-GCCcore-12.3.0.eb
    • Tcl-8.6.13-GCCcore-13.1.0.eb
    • Tcl-8.6.13-GCCcore-13.2.0.eb
    • Tcl-8.6.9-GCCcore-8.2.0.eb
    • Tcl-8.6.9-GCCcore-8.3.0.eb
  • [x] Tkinter: unrecognized options: --with-tcltk-includes, --with-tcltk-libs, fixed in #3087
    • [x] Tkinter-3.11.3-GCCcore-12.3.0.eb
    • [x] Tkinter-3.11.5-GCCcore-13.2.0.eb
  • [x] Yambo-5.2.dev-20230512-intel-2021b.eb Not an official EC

Flamefire avatar Oct 25 '23 09:10 Flamefire

@Flamefire Since this could break existing easyconfigs, we should target this for EasyBuild 5.0 imho, which gives us some leeway (keep in mind that we don't control all easyconfigs out there).

Figuring out which easyconfigs need fixing in our central easyconfigs repository is going to be really painful, but it's an exercise we need to do anyway for EasyBuild 5.0 for various other reasons (and it's a lot less painful since we've archived a large amount of old easyconfigs in the 5.0.x branches aleady).

So, can you re-target this to 5.0.x branch?

boegel avatar Oct 27 '23 09:10 boegel

I added #3026 for the 4.x branch to (only) show a well-visible warning and included that commit here upgrading it to an error in a 2nd commit (I guess you merge develop to 5.x regularly? That should help avoiding the conflict)

Flamefire avatar Oct 27 '23 11:10 Flamefire

@boegel I added the requested opt-out option. I felt that a bool is not enough as we might want to still show a warning but not fail the build. Hence I reused ERROR, WARN, IGNORE

Flamefire avatar Jan 05 '24 13:01 Flamefire

@Flamefire Can you look into fixing merge conflict?

boegel avatar Jan 15 '24 14:01 boegel

@Flamefire Can you look into fixing merge conflict?

Done. Was simply the run_shell_cmd change.

Flamefire avatar Jan 15 '24 15:01 Flamefire

As for documenting "ERROR": We already have "defaults to aborting the build" and the default is ERROR so IMO this is already sufficiently documented. The alternative "'ERROR' (the default) aborts the build with an error" or something like that sounds bulky and doesn't add much value as a) we already give the information that this is the default and b) I don't expect anyone to use this value but rather omit the EC param if that behavior is intended

Flamefire avatar Jan 22 '24 15:01 Flamefire

FYI: This found an actual issue for me: Python 3.11 moved from using configure args to env variables for TCL

Flamefire avatar Jan 23 '24 11:01 Flamefire

With the recent updates to the EasyConfigs this can now be merged. I did an extensive testing using the current develop ECs with the identified, potentially problematic ones, see https://github.com/easybuilders/easybuild-easyblocks/pull/3026#issuecomment-1986100888

Note that Tcl still has the configure warning in the log but it happens during make i.e. the build step. So there is some internal issue with Tcl but as it doesn't happen in the configure step it does not trigger this check. Hence it is not an issue for us.

Flamefire avatar Mar 11 '24 08:03 Flamefire

@Flamefire I've updated the overview in the PR description, it seems like there are a couple of unresolved cases still (and some PRs that should get merged soon).

A lot of progress has been made though, it definitely looks like we'll be able to get this change included in EasyBuild v5.0 now, thanks a lot for your efforts on this!

boegel avatar Mar 11 '24 13:03 boegel

MESS-0.1.6-foss-2019b.eb requires SLATEC but the slatec_src.tgz is not available, at least not the right version (sha df009d9ef9c18aae06ce68711b1ae108d3533b4f174582c3cbea0915c4fdfe01) Given the 2019b toolchain I'd let that slide. Or does anyone have that source available somewhere?

git-annex-10.20230802-GCCcore-12.2.0.eb uses MakeCp and I can't find any issue in the log

gap-4.12.2-foss-2022a.eb Shows the warning only in the build step because the internal command passes this to all dependencies and one doesn't know about it.

bigdft-suite-1.9.1-foss-2021b.eb is BigDFT and doesn't exist It was renamed in the PR that would have added it: https://github.com/easybuilders/easybuild-easyconfigs/pull/15860

Yambo-5.2.dev-20230512-intel-2021b.eb must be some custom EC of yours. We only have Yambo-5.1.2-intel-2021b.eb and that installs without issues.

Flamefire avatar Mar 11 '24 16:03 Flamefire

I finished checking the list. In https://github.com/easybuilders/easybuild-easyblocks/pull/3026#issuecomment-1986100888 I also tested everything on our system that potentially had this issue and all installed fine.

Hence I'd strongly suggest to merge both PRs: The deprecation warning for 4.x and this for 5.x

Flamefire avatar Mar 12 '24 08:03 Flamefire