easybuild icon indicating copy to clipboard operation
easybuild copied to clipboard

Issues when compiling OpenMPI with a job

Open gkaf89 opened this issue 8 months ago • 3 comments

I am trying to compile OpenMPI-4.1.6-GCC-13.2.0.eb with a GC3Pie job with EasyBuild 4.9.1. Everything works without issue apart from the compilation of MPI itself. The compilation fails with the message:

...
--- MCA component pmix:pmix3x (m4 configuration macro)
checking for MCA component pmix:pmix3x compile mode... dso
configure: WARNING: Found configure shell variable clash at line 175707!
configure: WARNING: OPAL_VAR_SCOPE_PUSH called on "PMIX_VERSION",
configure: WARNING: but it is already defined with value "4.2.9"
configure: WARNING: This usually indicates an error in configure.
configure: error: Cannot continue
 (at easybuild/iris/2023b/gpu/software/EasyBuild/4.9.1/lib/python3.11/site-packages/easybuild/tools/run.py:682 in parse_cmd_output)
...

The compilation works without issues when I create an allocation with salloc and build MPI in a local process in the allocation.

Is this a known issue?

Configuration details

The configuration used for the build job is:

[basic]

repositorypath       = /work/projects/software_stack_alpha/easybuild/iris/2023b/gpu/ebfiles_repo
robot-paths          = %(DEFAULT_ROBOT_PATHS)s
robot                = %(repositorypath)s:/work/projects/software_stack_alpha/backup/easybuild/easyconfigs

[config]

module-naming-scheme = CategorizedModuleNamingScheme
prefix               = /work/projects/software_stack_alpha/easybuild/iris/2023b/gpu
buildpath            = /tmp/easybuild/iris/2023b/gpu/build
containerpath        = /work/projects/software_stack_alpha/easybuild/iris/2023b/gpu/containers
installpath          = /work/projects/software_stack_alpha/easybuild/iris/2023b/gpu
packagepath          = /work/projects/software_stack_alpha/easybuild/iris/2023b/gpu/packages
sourcepath           = /work/projects/software_stack_alpha/easybuild/iris/2023b/gpu/sources

job                  = True
job-backend          = GC3Pie
tmpdir           = /work/projects/software_stack_alpha/tmp

job-backend-config   = configuration/GC3Pie/iris_gpu_gc3pie.cfg
job-output-dir       = ./logs
job-cores            = 7
job-polling-interval = 8
job-max-walltime     = 4
job-max-jobs         = 8

The contents of the configuration/GC3Pie/iris_gpu_gc3pie.cfg are:

[resource/slurm]
enabled = yes
type = slurm

# use settings below when running GC3Pie on the cluster front-end node
frontend = localhost
transport = local
auth = none

max_walltime = 2 days
# max # jobs ~= max_cores / max_cores_per_job
max_cores_per_job = 7
max_cores = 112
max_memory_per_core = 14200 MiB
architecture = x86_64

# to add non-std options or use SLURM tools located outside of
# the default PATH, use the following:
sbatch = /usr/bin/sbatch
  --mail-type=FAIL
  --partition=all
  --qos=admin
  --ntasks=1
  --cpus-per-task=7
  --gpus-per-task=1

The target system in the GPU partition of the Iris computer at the University of Luxembourg.

gkaf89 avatar Jun 13 '24 15:06 gkaf89