easybuild-easyblocks icon indicating copy to clipboard operation
easybuild-easyblocks copied to clipboard

Published 20 hours ago • verified publisher icon easybuilders

use relative paths to object files when compiling shared libraries in the OpenFOAM easyblock

Open bedroge opened this issue 1 year ago • 5 comments
trafficstars

In EESSI we ran into the following error when compiling OpenFOAM v2312 for Intel Skylake (see https://github.com/EESSI/software-layer/pull/563#issuecomment-2180005389):

make: /bin/sh: Argument list too long
make: *** [/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/wmake/makefiles/general:210: /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so] Error 127

This worked fine for other CPU targets, but due to the CPU target names that we use, Skylake has the longest installation path ("intel/skylake_avx512"). Apparently, this makes the entire command for building the shared library libOpenFOAM.so just a bit too long. This command includes a large number (~600) of absolute paths to object files:

OMPI_CXX="g++" mpicxx -m64 -pthread -DOPENFOAM=2312 -DWM_DP -DWM_LABEL_SIZE=32 -Wall -Wextra -Wold-style-cast -Wnon-virtual-dtor -Wno-unused-parameter -Wno-invalid-offsetof -Wno-attributes -Wno-unknown-pragmas -O2 -fno-tree-vectorize -march=native -fno-math-errno -std=c++14 -fuse-ld=bfd  -DNoRepository -ftemplate-depth-100  -I/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/build/linux64GccDPInt32Opt/src/OpenFOAM -DHAVE_LIBZ -DHAVE_EXTRAE -iquote. -IlnInclude -I/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/src/OpenFOAM/lnInclude -I/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/src/OSspecific/POSIX/lnInclude   -fPIC -shared -Xlinker --add-needed -Xlinker --no-as-needed  
/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/build/linux64GccDPInt32Opt/src/OpenFOAM/global/foamConfig.o
/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/build/linux64GccDPInt32Opt/src/OpenFOAM/global/globals.o
<about 600 more of these /paths/to/some.o> 
-L/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/platforms/linux64GccDPInt32Opt/lib \
    /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/platforms/linux64GccDPInt32Opt/lib/libOSspecific.o -lz -L/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/platforms/linux64GccDPInt32Opt/lib/dummy -lPstream  -o /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/OpenFOAM/v2312-foss-2023a/OpenFOAM-v2312/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so

This PR fixes the issue by replacing the absolute paths to all these object files by relative ones. It does that by replacing the $WM_PROJECT_DIR (main installation path of this OpenFOAM version) part of the path by the relative path (relative to the current working dir) to this main installation directory for every object file. This means that the ~600 paths will then look something like:

../../../build/linux64GccDPInt32Opt/src/Pstream/dummy/UPstream.o

And this greatly reduces the length of this entire command, which made it compile without issues for our Skylake installation path.

Since the same code seems to be part of both different OpenFOAM forks and in all the versions that I checked, I haven't restricted this to a specific fork/version.

bedroge avatar Jul 09 '24 13:07 bedroge

Actually, this makefile was introduced in 4.x for the .org fork. For older versions, the file does not exist, but in that case the change just won't be applied (as it checks if the file exists), which should be fine.

For the .com version, this file should be present for all versions.

bedroge avatar Jul 09 '24 14:07 bedroge

Successful build with the EESSI bot for skylake: https://github.com/EESSI/software-layer/pull/563#issuecomment-2217797794

bedroge avatar Jul 09 '24 19:07 bedroge

@boegelbot please test @ generoso CORE_CNT=16 EB_ARGS="--installpath=/tmp/$USER/pr3388 OpenFOAM-8-foss-2020a.eb OpenFOAM-9-foss-2021a.eb OpenFOAM-10-foss-2022a.eb OpenFOAM-11-foss-2023a.eb OpenFOAM-v2106-foss-2021a.eb OpenFOAM-v2112-foss-2021b.eb OpenFOAM-v2206-foss-2022a.eb OpenFOAM-v2306-foss-2022b.eb OpenFOAM-v2312-foss-2023a.eb"

bedroge avatar Jul 09 '24 20:07 bedroge

@bedroge: Request for testing this PR well received on login1

PR test command 'EB_PR=3388 EB_ARGS="--installpath=/tmp/$USER/pr3388 OpenFOAM-8-foss-2020a.eb OpenFOAM-9-foss-2021a.eb OpenFOAM-10-foss-2022a.eb OpenFOAM-11-foss-2023a.eb OpenFOAM-v2106-foss-2021a.eb OpenFOAM-v2112-foss-2021b.eb OpenFOAM-v2206-foss-2022a.eb OpenFOAM-v2306-foss-2022b.eb OpenFOAM-v2312-foss-2023a.eb" EB_CONTAINER= EB_REPO=easybuild-easyblocks /opt/software/slurm/bin/sbatch --job-name test_PR_3388 --ntasks="16" ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 13909

Test results coming soon (I hope)...

- notification for comment with ID 2218652517 processed

Message to humans: this is just bookkeeping information for me, it is of no use to you (unless you think I have a bug, which I don't).

boegelbot avatar Jul 09 '24 20:07 boegelbot

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • SUCCESS OpenFOAM-8-foss-2020a.eb
  • SUCCESS OpenFOAM-9-foss-2021a.eb
  • SUCCESS OpenFOAM-10-foss-2022a.eb
  • SUCCESS OpenFOAM-11-foss-2023a.eb
  • SUCCESS OpenFOAM-v2106-foss-2021a.eb
  • SUCCESS OpenFOAM-v2112-foss-2021b.eb
  • SUCCESS OpenFOAM-v2206-foss-2022a.eb
  • SUCCESS OpenFOAM-v2306-foss-2022b.eb
  • SUCCESS OpenFOAM-v2312-foss-2023a.eb

Build succeeded for 9 out of 9 (9 easyconfigs in total) cnx1 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8 See https://gist.github.com/boegelbot/b5e53db5955ea2e03c35574f917e7ad3 for a full test report.

boegelbot avatar Jul 10 '24 06:07 boegelbot