easybuild-easyblocks icon indicating copy to clipboard operation
easybuild-easyblocks copied to clipboard

Published 20 hours ago • verified publisher icon easybuilders

deprecate unknown configure args in `ConfigureMake` easyblock

Open Flamefire opened this issue 2 years ago • 9 comments
trafficstars

More compatible version of #3025 for 4.x to only deprecate and verbosely warn about unknown configure args

Flamefire avatar Oct 27 '23 11:10 Flamefire

This makes sense to include in EasyBuild 4.x, but we should still try and do a sweep across all easyconfigs using ConfigureMake (or an easyblock that derives from ConfigureMake) so see how often this would start producing scary warnings...

That's a huge list of easyconfigs, so it's a pretty painful exercise, but a necessary one imho, unless we're willing to accept exposing people to these warnings (which may help in getting the problems fixed, I guess).

boegel avatar Nov 08 '23 08:11 boegel

That's a huge list of easyconfigs, so it's a pretty painful exercise, but a necessary one imho, unless we're willing to accept exposing people to these warnings (which may help in getting the problems fixed, I guess).

I'd opt for the latter. I actually run the EasyBlock which fails hard instead on our cluster such that those things actually make me fix it. A scary warning to convince more people to cleanup the ECs sounds like a good compromise to get this big task done.

Doing a bulk fix is going to be hard as even though you can use --stop=configure for this you'd need to have dependencies installed.

Flamefire avatar Nov 08 '23 10:11 Flamefire

I did a quick scan for "unrecognized options" in my set of installations, and the installation logs for the modules below had hits.

Most of these were done as a part of the regression test for EasyBuild v4.8.2, so recent installations, but some are older (marked with (*)) and those may already be fixed.

  • BigDFT/1.9.1-foss-2021b: "unrecognized options: --disable-Werror, --with-yaml-path"
  • FriBidi/1.0.10-GCCcore-10.2.0: "unrecognized options: --disable-docs"
  • FriBidi/1.0.10-GCCcore-10.3.0 (same as above)
  • FriBidi/1.0.10-GCCcore-11.2.0 (same as above)
  • FriBidi/1.0.12-GCCcore-12.2.0 (same as above)
  • FriBidi/1.0.12-GCCcore-12.3.0 (same as above)
  • FriBidi/1.0.5-GCCcore-8.3.0 (*) (same as above)
  • FriBidi/1.0.9-GCCcore-9.3.0 (*) (same as above)
  • GDAL/3.0.2-foss-2019b-Python-3.7.4 (*): "unrecognized options: --with-libgeotiff"
  • GDAL/3.0.2-intel-2019b-Python-3.7.4 (*) (same as above)
  • GDAL/3.0.4-foss-2020a-Python-3.8.2 (*) (same as above)
  • GDAL/3.5.0-foss-2022a (*): "unrecognized options: --with-jasper"
  • GRASS/8.2.0-foss-2021b: "unrecognized options: --enable-64bit, --with-ffmpeg, --with-ffmpeg-libs, --with-ffmpeg-includes, --without-glw, --with-lapack-lib, --with-blas-lib, --with-proj, --with-python, --with-spatialite, --with-zlib, --with-ibzlib-includes"
  • groff/1.23.0-GCCcore-13.2.0: "unrecognized options: --with-doc"
  • GTK+/3.24.13-GCCcore-8.3.0 (*): "unrecognized options: --disable-visibility"
  • GTK+/3.24.23-GCCcore-10.2.0 (*) (same as above)
  • GTK3/3.24.29-GCCcore-10.3.0 (same as above)
  • GTK3/3.24.31-GCCcore-11.2.0 (same as above)
  • GTK3/3.24.35-GCCcore-12.2.0 (same as above)
  • hwloc/2.2.0-GCCcore-10.2.0 (*): "unrecognized options: --enable-libnuma"
  • hwloc/2.2.0-GCCcore-9.3.0 (*) (same as above)
  • hwloc/2.4.1-GCCcore-10.3.0 (same as above)
  • hwloc/2.5.0-GCCcore-11.2.0 (same as above)
  • hwloc/2.7.1-GCCcore-11.3.0 (same as above)
  • hwloc/2.8.0-GCCcore-12.2.0 (same as above)
  • hwloc/2.9.1-GCCcore-12.3.0 (same as above)
  • hwloc/2.9.2-GCCcore-13.2.0 (same as above)
  • libfdf/0.2.2-GCC-10.3.0-serial: "unrecognized options: --without-mpi"
  • libfdf/0.2.2-GCC-11.2.0-serial (same as above)
  • Libint/2.6.0-GCC-10.3.0-lmax-6-cp2k: "unrecognized options: --with-cxxgen-optflags"
  • Libint/2.6.0-iimpi-2021a-lmax-6-cp2k (same as above)
  • Libint/2.6.0-gompi-2020a-lmax-6-cp2k (*) (same as above)
  • LibTIFF/4.1.0-GCCcore-10.2.0 (*): "unrecognized options: --disable-libdeflate"
  • LibTIFF/4.1.0-GCCcore-9.3.0 (*) (same as above)
  • MEME/5.4.1-GCC-10.3.0: "unrecognized options: --with-python3"
  • MESS/0.1.6-foss-2019b (*): "unrecognized options: --enable-fortran"
  • OpenMPI/4.0.6-GCC-10.3.0: "unrecognized options: --with-ucc"
  • OpenMPI/4.0.7-GCC-10.3.0 (same as above)
  • OpenMPI/4.1.1-GCC-10.3.0 (same as above)
  • OpenMPI/4.1.1-GCC-11.2.0 (same as above)
  • OpenMPI/4.1.2-GCC-11.2.0 (same as above)
  • Tcl/8.6.10-GCCcore-10.2.0 (*): "unrecognized options: --with-tclinclude, --enable-shared"
  • Tcl/8.6.10-GCCcore-9.3.0 (*) (same as above)
  • Tcl/8.6.11-GCCcore-10.3.0 (same as above)
  • Tcl/8.6.11-GCCcore-11.2.0 (same as above)
  • Tcl/8.6.12-GCCcore-11.3.0 (same as above)
  • Tcl/8.6.12-GCCcore-12.2.0 (same as above)
  • Tcl/8.6.13-GCCcore-12.3.0 (same as above)
  • Tcl/8.6.13-GCCcore-13.2.0 (same as above)
  • Tkinter/3.11.3-GCCcore-12.3.0: "unrecognized options: --with-tcltk-includes, --with-tcltk-libs"
  • Tkinter/3.11.5-GCCcore-13.2.0 (same as above)

Although not a blocker for merging this PR, it would be good to have these easyconfigs updated so the warning won't present itself when installing these...

boegel avatar Nov 08 '23 11:11 boegel

I checked my recent tree and also found the Tcl issue. However it looks like it is some configure called by configure passing those flags as we only do ./configure --prefix=/software/Tcl/8.6.12-GCCcore-11.3.0 --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --enable-threads EXTRA_INSTALL="install-private-headers"

Flamefire avatar Nov 08 '23 16:11 Flamefire

I did a scan for "unrecognized options" in all installations we have on generoso (9,991 installations), which revealed these:

BigDFT/1.9.1-foss-2021b
configure: WARNING: unrecognized options: --disable-Werror, --enable-mpi, --with-yaml-path, --with-ext-linalg
configure: WARNING: unrecognized options: --disable-Werror
configure: WARNING: unrecognized options: --disable-Werror, --with-yaml-path
configure: WARNING: unrecognized options: --disable-Werror, --with-yaml-path, --with-ext-linalg

FFLAS-FFPACK/2.5.0-gfbf-2022a
configure: WARNING: unrecognized options: --with-gmp, --with-givaro

FriBidi/1.0.10-GCCcore-10.2.0
FriBidi/1.0.10-GCCcore-10.3.0
FriBidi/1.0.10-GCCcore-11.2.0
FriBidi/1.0.12-GCCcore-11.3.0
FriBidi/1.0.12-GCCcore-12.2.0
FriBidi/1.0.12-GCCcore-12.3.0
FriBidi/1.0.13-GCCcore-13.2.0
FriBidi/1.0.5-GCCcore-7.3.0
FriBidi/1.0.5-GCCcore-8.2.0
FriBidi/1.0.5-GCCcore-8.3.0
FriBidi/1.0.9-GCCcore-9.3.0
configure: WARNING: unrecognized options: --disable-docs

GDAL/2.2.3-foss-2018b-Python-2.7.15
GDAL/2.2.3-foss-2018b-Python-3.6.6
GDAL/3.0.0-foss-2019a-Python-3.7.2
GDAL/3.0.2-foss-2019b-Python-3.7.4
GDAL/3.0.2-intel-2019b-Python-3.7.4
GDAL/3.0.4-foss-2020a-Python-3.8.2
configure: WARNING: unrecognized options: --with-libgeotiff

GDAL/3.5.0-foss-2022a
configure: WARNING: unrecognized options: --with-jasper

GRASS/8.2.0-foss-2021b
configure: WARNING: unrecognized options: --enable-64bit, --with-ffmpeg, --with-ffmpeg-libs, --with-ffmpeg-includes, --without-glw, --with-lapack-lib, --with-blas-lib, --with-proj, --with-python, --with-spatialite, --with-zlib, --with-ibzlib-includes

GTK+/3.24.13-GCCcore-8.3.0
GTK+/3.24.17-GCCcore-9.3.0
GTK+/3.24.23-GCCcore-10.2.0
GTK+/3.24.8-GCCcore-8.2.0
configure: WARNING: unrecognized options: --disable-visibility

GTK3/3.24.29-GCCcore-10.3.0
GTK3/3.24.31-GCCcore-11.2.0
GTK3/3.24.33-GCCcore-11.3.0
GTK3/3.24.35-GCCcore-12.2.0
configure: WARNING: unrecognized options: --disable-visibility

GnuTLS/3.7.2-GCCcore-10.3.0
configure: WARNING: unrecognized options: --without-tpm2

HDF-EOS2/3.0-GCCcore-11.3.0
configure: WARNING: unrecognized options: --enable-install-include

LibTIFF/4.1.0-GCCcore-10.2.0
LibTIFF/4.1.0-GCCcore-8.3.0
LibTIFF/4.1.0-GCCcore-9.3.0
configure: WARNING: unrecognized options: --disable-libdeflate

Libint/2.6.0-GCC-10.2.0-lmax-6-cp2k
Libint/2.6.0-GCC-10.3.0-lmax-6-cp2k
Libint/2.6.0-GCC-11.3.0-lmax-6-cp2k
Libint/2.6.0-gompi-2020a-lmax-6-cp2k
Libint/2.6.0-iccifort-2020.4.304-lmax-6-cp2k
Libint/2.6.0-iimpi-2021a-lmax-6-cp2k
configure: WARNING: unrecognized options: --with-cxxgen-optflags

LinBox/1.7.0-gfbf-2022a
configure: WARNING: unrecognized options: --with-fflas-ffpack, --with-givaro, --enable-openmp

MDSplus/7.96.12-GCCcore-9.3.0
configure: WARNING: unrecognized options: --disable-hdf5

MEME/5.4.1-GCC-10.3.0
configure: WARNING: unrecognized options: --with-python3
configure: WARNING: unrecognized options: --with-python3

MEME/5.4.1-GCC-11.2.0
configure: WARNING: unrecognized options: --with-python3
configure: WARNING: unrecognized options: --with-python3

MIRA/5.0rc2-foss-2020b
configure: WARNING: unrecognized options: --with-tcmalloc-dir

OpenMPI/4.1.1-intel-compilers-2021.4.0
configure: WARNING: unrecognized options: --with-ucc

Tcl/8.6.10-GCCcore-10.2.0
Tcl/8.6.10-GCCcore-9.3.0
Tcl/8.6.11-GCCcore-10.3.0
Tcl/8.6.11-GCCcore-11.2.0
Tcl/8.6.12-GCCcore-11.3.0
Tcl/8.6.12-GCCcore-12.2.0
Tcl/8.6.13-GCCcore-12.3.0
Tcl/8.6.13-GCCcore-13.1.0
Tcl/8.6.13-GCCcore-13.2.0
Tcl/8.6.9-GCCcore-8.2.0
Tcl/8.6.9-GCCcore-8.3.0
configure: WARNING: unrecognized options: --with-tclinclude, --enable-shared

Tkinter/3.11.3-GCCcore-12.3.0
configure: WARNING: unrecognized options: --with-tcltk-includes, --with-tcltk-libs

Yambo/5.2.dev-20230512-intel-2021b
configure: WARNING: unrecognized options: --with-netcdf-lib, --with-netcdf-include

bigdft-suite/1.9.1-foss-2021b
configure: WARNING: unrecognized options: --disable-Werror, --enable-mpi, --with-ext-linalg, --with-yaml-path
configure: WARNING: unrecognized options: --disable-Werror
configure: WARNING: unrecognized options: --disable-Werror, --with-yaml-path
configure: WARNING: unrecognized options: --disable-Werror, --with-ext-linalg, --with-yaml-path

gap/4.12.2-foss-2022a
configure: WARNING: unrecognized options: --with-gaproot

git-annex/10.20230802-GCCcore-12.2.0
configure: WARNING: unrecognized options: --with-compiler

groff/1.23.0-GCCcore-13.2.0
configure: WARNING: unrecognized options: --with-doc

hwloc/2.0.2-GCCcore-8.2.0
hwloc/2.0.3-GCCcore-8.3.0
hwloc/2.1.0-GCCcore-9.2.0
hwloc/2.2.0-GCCcore-10.2.0
hwloc/2.2.0-GCCcore-9.3.0
hwloc/2.4.1-GCCcore-10.3.0
hwloc/2.5.0-GCCcore-11.2.0
hwloc/2.7.1-GCCcore-11.3.0
hwloc/2.8.0-GCCcore-12.2.0
hwloc/2.9.1-GCCcore-12.3.0
hwloc/2.9.2-GCCcore-13.2.0
configure: WARNING: unrecognized options: --enable-libnuma

libfdf/0.2.2-GCC-10.2.0-serial
libfdf/0.2.2-GCC-10.3.0-serial
libfdf/0.2.2-GCC-11.2.0-serial
libfdf/0.2.2-iccifort-2020.4.304-serial
libfdf/0.2.2-intel-compilers-2021.2.0-serial
libfdf/0.2.2-intel-compilers-2021.4.0-serial
configure: WARNING: unrecognized options: --without-mpi

boegel avatar Jan 03 '24 19:01 boegel

That's quite a few and we may want to fix them (for the non-ancient and at least easy ones) for EB 5.
Given the list above and knowing that removing that param has no effect at all we might also just fix those without doing a build.

But: Do we still need to consider/handle EB 4.x PRs after 4.9.0 was released?

Flamefire avatar Jan 04 '24 08:01 Flamefire

@boegel I compiled as list of EC filenames from your 2 comments and mentioned in #3025 and am running tests building those using develop. The resulting PRs should be all findable by https://github.com/easybuilders/easybuild-easyconfigs/pulls?q=is%3Apr+author%3AFlamefire++configure

Flamefire avatar Feb 29 '24 12:02 Flamefire

Test report by @Flamefire

Overview of tested easyconfigs (in order)

  • SUCCESS BigDFT-1.9.1-foss-2021b.eb
  • SUCCESS FFLAS-FFPACK-2.5.0-gfbf-2022a.eb
  • SUCCESS FriBidi-1.0.10-GCCcore-10.2.0.eb
  • SUCCESS FriBidi-1.0.10-GCCcore-10.3.0.eb
  • SUCCESS FriBidi-1.0.10-GCCcore-11.2.0.eb
  • SUCCESS FriBidi-1.0.12-GCCcore-11.3.0.eb
  • SUCCESS FriBidi-1.0.12-GCCcore-12.2.0.eb
  • SUCCESS FriBidi-1.0.12-GCCcore-12.3.0.eb
  • SUCCESS FriBidi-1.0.13-GCCcore-13.2.0.eb
  • SUCCESS FriBidi-1.0.5-GCCcore-7.3.0.eb
  • SUCCESS FriBidi-1.0.5-GCCcore-8.2.0.eb
  • SUCCESS FriBidi-1.0.5-GCCcore-8.3.0.eb
  • SUCCESS FriBidi-1.0.9-GCCcore-9.3.0.eb
  • SUCCESS gap-4.12.2-foss-2022a.eb
  • SUCCESS GDAL-2.2.3-foss-2018b-Python-2.7.15.eb
  • SUCCESS GDAL-3.0.0-foss-2019a-Python-3.7.2.eb
  • SUCCESS GDAL-3.0.2-foss-2019b-Python-3.7.4.eb
  • SUCCESS GDAL-3.5.0-foss-2022a.eb
  • SUCCESS git-annex-10.20230802-GCCcore-12.2.0.eb
  • SUCCESS GnuTLS-3.7.2-GCCcore-10.3.0.eb
  • SUCCESS GRASS-8.2.0-foss-2021b.eb
  • SUCCESS groff-1.23.0-GCCcore-13.2.0.eb
  • SUCCESS GTK+-3.24.13-GCCcore-8.3.0.eb
  • SUCCESS GTK+-3.24.17-GCCcore-9.3.0.eb
  • SUCCESS GTK+-3.24.23-GCCcore-10.2.0.eb
  • SUCCESS GTK+-3.24.8-GCCcore-8.2.0.eb
  • SUCCESS GTK3-3.24.29-GCCcore-10.3.0.eb
  • SUCCESS GTK3-3.24.31-GCCcore-11.2.0.eb
  • SUCCESS GTK3-3.24.33-GCCcore-11.3.0.eb
  • SUCCESS GTK3-3.24.35-GCCcore-12.2.0.eb
  • SUCCESS HDF-EOS2-3.0-GCCcore-11.3.0.eb
  • SUCCESS hwloc-2.0.2-GCCcore-8.2.0.eb
  • SUCCESS hwloc-2.0.3-GCCcore-8.3.0.eb
  • SUCCESS hwloc-2.1.0-GCCcore-9.2.0.eb
  • SUCCESS hwloc-2.2.0-GCCcore-10.2.0.eb
  • SUCCESS hwloc-2.2.0-GCCcore-9.3.0.eb
  • SUCCESS hwloc-2.4.1-GCCcore-10.3.0.eb
  • SUCCESS hwloc-2.5.0-GCCcore-11.2.0.eb
  • SUCCESS hwloc-2.7.1-GCCcore-11.3.0.eb
  • SUCCESS hwloc-2.8.0-GCCcore-12.2.0.eb
  • SUCCESS hwloc-2.9.1-GCCcore-12.3.0.eb
  • SUCCESS hwloc-2.9.2-GCCcore-13.2.0.eb
  • SUCCESS libfdf-0.2.2-GCC-10.2.0-serial.eb
  • SUCCESS libfdf-0.2.2-GCC-10.3.0-serial.eb
  • SUCCESS libfdf-0.2.2-GCC-11.2.0-serial.eb
  • SUCCESS Libint-2.6.0-GCC-10.2.0-lmax-6-cp2k.eb
  • SUCCESS Libint-2.6.0-GCC-10.3.0-lmax-6-cp2k.eb
  • SUCCESS Libint-2.6.0-gompi-2020a-lmax-6-cp2k.eb
  • SUCCESS Libint-2.7.2-GCC-11.3.0-lmax-6-cp2k.eb
  • SUCCESS LibTIFF-4.1.0-GCCcore-10.2.0.eb
  • SUCCESS LibTIFF-4.1.0-GCCcore-8.3.0.eb
  • SUCCESS LibTIFF-4.1.0-GCCcore-9.3.0.eb
  • SUCCESS LinBox-1.7.0-gfbf-2022a.eb
  • SUCCESS MDSplus-7.96.12-GCCcore-9.3.0.eb
  • SUCCESS MEME-5.4.1-GCC-10.3.0.eb
  • SUCCESS MIRA-5.0rc2-foss-2020b.eb
  • SUCCESS OpenMPI-4.0.6-GCC-10.3.0.eb
  • SUCCESS OpenMPI-4.0.7-GCC-10.3.0.eb
  • SUCCESS OpenMPI-4.1.1-GCC-10.3.0.eb
  • SUCCESS OpenMPI-4.1.1-GCC-11.2.0.eb
  • SUCCESS OpenMPI-4.1.2-GCC-11.2.0.eb
  • SUCCESS Tcl-8.6.10-GCCcore-10.2.0.eb
  • SUCCESS Tcl-8.6.10-GCCcore-9.3.0.eb
  • SUCCESS Tcl-8.6.11-GCCcore-10.3.0.eb
  • SUCCESS Tcl-8.6.11-GCCcore-11.2.0.eb
  • SUCCESS Tcl-8.6.12-GCCcore-11.3.0.eb
  • SUCCESS Tcl-8.6.12-GCCcore-12.2.0.eb
  • SUCCESS Tcl-8.6.13-GCCcore-12.3.0.eb
  • SUCCESS Tcl-8.6.13-GCCcore-13.1.0.eb
  • SUCCESS Tcl-8.6.13-GCCcore-13.2.0.eb
  • SUCCESS Tcl-8.6.9-GCCcore-8.2.0.eb
  • SUCCESS Tcl-8.6.9-GCCcore-8.3.0.eb
  • SUCCESS Tkinter-3.11.3-GCCcore-12.3.0.eb
  • SUCCESS Tkinter-3.11.5-GCCcore-13.2.0.eb
  • SUCCESS GDAL-3.0.4-foss-2020a-Python-3.8.2.eb

Build succeeded for 75 out of 75 (75 easyconfigs in total) n1358 - Linux RHEL 8.7 (Ootpa), x86_64, Intel(R) Xeon(R) Platinum 8470 (icelake), Python 3.8.13 See https://gist.github.com/Flamefire/7248cfe5c3a8bc6c24932231bc839e97 for a full test report.

Flamefire avatar Mar 08 '24 17:03 Flamefire

@boegel The above test report was done with a temporary change to issue an error instead of a warning. As you can see all ECs you identified are now fixed. So I'd say this is ready now.

Flamefire avatar Mar 11 '24 08:03 Flamefire