OpenMPI V5.0.0 --with-pmix=internal, Make returns with Errors

[W-Wuxian]

Thank you for taking the time to submit an issue!

What version of Open MPI are you using? (e.g., v3.0.5, v4.0.2, git branch name and hash, etc.)

OpenMPI v5.0.0

Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)

module purge
module load gcc-10.2.0
../configure CFLAGS="-std=c99 -D_BSD_SOURCE" --prefix=$WHERE_TO_INSTALL_OPENMPI_WITH_GCC --with-pmix=internal --with-hwloc=internal

Please describe the system on which you are running

• Operating system/version: uname -a Linux 3.10.0-1160.102.1.el7.x86_64 # 1 S M P Tue Oct 17 15:42:21 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux CentOS 7.9 64 bits

• Computer hardware: 4 Intel Xeon Platinium 24 cores 4.40GHz 385 Go

• Network type: None

---

Details of the problem

At make time

Without -D_BSD_SOURCE

make[3] : on entre dans le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/libevent-2.1.12-stable »
  CC       buffer.lo
  CC       bufferevent.lo
  CC       bufferevent_filter.lo
  CC       bufferevent_pair.lo
  CC       bufferevent_ratelim.lo
  CC       bufferevent_sock.lo
  CC       event.lo
  CC       evmap.lo
  CC       evthread.lo
  CC       evutil.lo
  CC       evutil_rand.lo
  CC       evutil_time.lo
  CC       listener.lo
  CC       log.lo
  CC       strlcpy.lo
  CC       select.lo
  CC       poll.lo
  CC       epoll.lo
  CC       signal.lo
  CC       evdns.lo
  CC       event_tagging.lo
  CC       evrpc.lo
  CC       http.lo
  CC       sample/event-read-fifo.o
  CC       sample/hello-world.o
  CC       sample/http-server.o
  CC       sample/http-connect.o
  CC       sample/signal-test.o
  CC       sample/time-test.o
  CCLD     libevent_core.la
  CC       evthread_pthread.lo
sample/time-test.c: In function ‘timeout_cb’:
sample/time-test.c:50:2: attention : implicit declaration of function ‘timersub’ [-Wimplicit-function-declaration]
  evutil_timersub(&newtime, &lasttime, &difference);
  ^
sample/time-test.c:60:3: attention : implicit declaration of function ‘timerclear’ [-Wimplicit-function-declaration]
   evutil_timerclear(&tv);
   ^
  CC       sample/dns-example.o
  CCLD     libevent.la
  CCLD     libevent_extra.la
  CCLD     libevent_pthreads.la
  CCLD     sample/dns-example
  CCLD     sample/event-read-fifo
  CCLD     sample/hello-world
  CCLD     sample/http-server
  CCLD     sample/http-connect
  CCLD     sample/signal-test
  CCLD     sample/time-test
sample/time-test.o: In function `timeout_cb':
time-test.c:(.text+0x44): undefined reference to `timersub'
time-test.c:(.text+0xc0): undefined reference to `timerclear'
sample/time-test.o: In function `main':
time-test.c:(.text+0x18e): undefined reference to `timerclear'
collect2: error: ld returned 1 exit status
make[3]: *** [sample/time-test] Erreur 1
make[3]: *** Attente des tâches non terminées....
make[3] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/libevent-2.1.12-stable »
make[2]: *** [all] Erreur 2
make[2] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/libevent-2.1.12-stable »
make[1]: *** [all-recursive] Erreur 1
make[1] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party »
make: *** [all-recursive] Erreur 1

Without -std=c99

make[4] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src/mca/ptl »
Making all in mca/common/dstore
make[4] : on entre dans le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src/mca/common/dstore »
  CC       dstore_base.lo
  CC       dstore_segment.lo
../../../../../../../3rd-party/openpmix/src/mca/common/dstore/dstore_base.c: In function ‘_esh_ns_track_cleanup’:
../../../../../../../3rd-party/openpmix/src/mca/common/dstore/dstore_base.c:380:5: erreur: ‘for’ loop initial declarations are only allowed in C99 mode
     for (int i = 0; i < size; i++) {
     ^
../../../../../../../3rd-party/openpmix/src/mca/common/dstore/dstore_base.c:380:5: note: use option -std=c99 or -std=gnu99 to compile your code
make[4]: *** [dstore_base.lo] Erreur 1
make[4]: *** Attente des tâches non terminées....
make[4] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src/mca/common/dstore »
make[3]: *** [all-recursive] Erreur 1
make[3] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src »
make[2]: *** [all-recursive] Erreur 1
make[2] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix »
make[1]: *** [all-recursive] Erreur 1
make[1] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party »
make: *** [all-recursive] Erreur 1

../../../../../3rd-party/openpmix/src/util/pmix_environ.c: In function ‘pmix_environ_merge_inplace’:
../../../../../3rd-party/openpmix/src/util/pmix_environ.c:109:5: erreur: ‘for’ loop initial declarations are only allowed in C99 mode
     for (size_t adders_idx = 0 ; adders[adders_idx] != NULL ; adders_idx++) {
     ^
../../../../../3rd-party/openpmix/src/util/pmix_environ.c:109:5: note: use option -std=c99 or -std=gnu99 to compile your code
../../../../../3rd-party/openpmix/src/util/pmix_environ.c: In function ‘pmix_getenv’:
../../../../../3rd-party/openpmix/src/util/pmix_environ.c:139:5: erreur: ‘for’ loop initial declarations are only allowed in C99 mode
     for (size_t env_idx = 0 ; env[env_idx] != NULL ; env_idx++) {
     ^
make[4]: *** [pmix_environ.lo] Erreur 1
make[4]: *** Attente des tâches non terminées....
../../../../../3rd-party/openpmix/src/util/pmix_fd.c: In function ‘pmix_close_open_file_descriptors’:
../../../../../3rd-party/openpmix/src/util/pmix_fd.c:263:5: erreur: ‘for’ loop initial declarations are only allowed in C99 mode
     for (int fd = 3; fd < fdmax; fd++) {
     ^
../../../../../3rd-party/openpmix/src/util/pmix_fd.c:263:5: note: use option -std=c99 or -std=gnu99 to compile your code
make[4]: *** [pmix_fd.lo] Erreur 1
make[4] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src/util »
make[3]: *** [all-recursive] Erreur 1
make[3] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src »
make[2]: *** [all-recursive] Erreur 1
make[2] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix »
make[1]: *** [all-recursive] Erreur 1
make[1] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party »
make: *** [all-recursive] Erreur 1

Using make VERBOSE=1 CFLAGS="-std=c99 -D_BSD_SOURCE" -j 4

make[4] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src/util »
Making all in mca/base
make[4] : on entre dans le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src/mca/base »
  CC       pmix_mca_base_close.lo
  CC       pmix_mca_base_component_find.lo
  CC       pmix_mca_base_component_compare.lo
  CC       pmix_mca_base_component_repository.lo
  CC       pmix_mca_base_components_open.lo
  CC       pmix_mca_base_components_close.lo
  CC       pmix_mca_base_components_select.lo
  CC       pmix_mca_base_list.lo
  CC       pmix_mca_base_open.lo
  CC       pmix_mca_base_var.lo
  CC       pmix_mca_base_var_enum.lo
  CC       pmix_mca_base_var_group.lo
In file included from /root/openmpi-5.0.0/3rd-party/openpmix/src/class/pmix_object.h:128:0,
                 from /root/openmpi-5.0.0/3rd-party/openpmix/src/class/pmix_list.h:79,
                 from /root/openmpi-5.0.0/3rd-party/openpmix/src/mca/base/pmix_base.h:32,
                 from ../../../../../../3rd-party/openpmix/src/mca/base/pmix_mca_base_var_enum.c:27:
../../../../../../3rd-party/openpmix/src/mca/base/pmix_mca_base_var_enum.c: In function ‘pmix_mca_base_var_enum_create_flag’:
../../../../../../3rd-party/openpmix/src/mca/base/pmix_mca_base_var_enum.c:315:18: erreur: ‘all_flags’ undeclared (first use in this function)
         assert(!(all_flags & flags[i].flag));
                  ^
../../../../../../3rd-party/openpmix/src/mca/base/pmix_mca_base_var_enum.c:315:18: note: each undeclared identifier is reported only once for each function it appears in
make[4]: *** [pmix_mca_base_var_enum.lo] Erreur 1
make[4]: *** Attente des tâches non terminées....
make[4] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src/mca/base »
make[3]: *** [all-recursive] Erreur 1
make[3] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src »
make[2]: *** [all-recursive] Erreur 1
make[2] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix »
make[1]: *** [all-recursive] Erreur 1
make[1] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party »
make: *** [all-recursive] Erreur 1

Rerunning with --enable-debug

make clean distclean
../configure CFLAGS="-std=c99 -D_BSD_SOURCE" --enable-debug --with-wrapper-cflags=CFLAGS --prefix=$WHERE_TO_INSTALL_OPENMPI_WITH_GCC --with-pmix=internal --with-hwloc=internal
make VERBOSE=1 CFLAGS="-std=c99 -D_BSD_SOURCE" -j 4

make[4] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src/mca/ptl/tool »
Making all in .
make[4] : on entre dans le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src »
  CC       threads/mutex.lo
  CC       threads/thread.lo
  CC       event/pmix_event_notification.lo
  CC       event/pmix_event_registration.lo
  CC       hwloc/pmix_hwloc.lo
  CC       hwloc/pmix_hwloc_datatype.lo
  CC       client/pmix_client.lo
  CC       client/pmix_client_fence.lo
  CC       client/pmix_client_get.lo
  CC       client/pmix_client_pub.lo
  CC       client/pmix_client_spawn.lo
  CC       client/pmix_client_connect.lo
  CC       client/pmix_client_group.lo
  CC       client/pmix_client_fabric.lo
  CC       client/pmix_client_topology.lo
  CC       client/pmix_client_convert.lo
  CC       server/pmix_server.lo
  CC       server/pmix_server_get.lo
  CC       server/pmix_server_ops.lo
  CC       runtime/pmix_finalize.lo
  CC       runtime/pmix_init.lo
  CC       runtime/pmix_params.lo
  CC       runtime/pmix_progress_threads.lo
  CC       tool/pmix_tool.lo
  CC       tool/pmix_tool_ops.lo
  CC       common/pmix_query.lo
  CC       common/pmix_strings.lo
  CC       common/pmix_log.lo
  CC       common/pmix_control.lo
  CC       common/pmix_data.lo
  CC       common/pmix_security.lo
  CC       common/pmix_iof.lo
  CC       common/pmix_attributes.lo
  CCLD     libpmix.la
make[4] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src »
Making all in mca/pcompress/zlib
make[4] : on entre dans le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src/mca/pcompress/zlib »
  CC       compress_zlib_component.lo
  CC       compress_zlib.lo
  CCLD     pmix_mca_pcompress_zlib.la
make[4] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src/mca/pcompress/zlib »
Making all in mca/prm/default
make[4] : on entre dans le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src/mca/prm/default »
  CC       prm_default_component.lo
  CC       prm_default.lo
  CCLD     pmix_mca_prm_default.la
make[4] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src/mca/prm/default »
Making all in mca/prm/slurm
make[4] : on entre dans le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src/mca/prm/slurm »
  CC       prm_slurm_component.lo
  CC       prm_slurm.lo
  CCLD     pmix_mca_prm_slurm.la
make[4] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src/mca/prm/slurm »
Making all in tools/pevent
make[4] : on entre dans le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src/tools/pevent »
  CC       pevent.o
  CCLD     pevent
../../../src/.libs/libpmix.so: undefined reference to `pthread_rwlockattr_init'
../../../src/.libs/libpmix.so: undefined reference to `pthread_mutexattr_settype'
../../../src/.libs/libpmix.so: undefined reference to `pthread_rwlock_rdlock'
../../../src/.libs/libpmix.so: undefined reference to `pthread_rwlock_init'
../../../src/.libs/libpmix.so: undefined reference to `pthread_getspecific'
../../../src/.libs/libpmix.so: undefined reference to `pthread_rwlock_destroy'
../../../src/.libs/libpmix.so: undefined reference to `pthread_rwlockattr_setpshared'
../../../src/.libs/libpmix.so: undefined reference to `pthread_key_create'
../../../src/.libs/libpmix.so: undefined reference to `pthread_rwlock_wrlock'
../../../src/.libs/libpmix.so: undefined reference to `pthread_kill'
../../../src/.libs/libpmix.so: undefined reference to `pthread_mutexattr_setpshared'
/root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/libevent-2.1.12-stable/.libs/libevent_pthreads.so: undefined reference to `pthread_mutex_trylock'
../../../src/.libs/libpmix.so: undefined reference to `pthread_setspecific'
../../../src/.libs/libpmix.so: undefined reference to `pthread_rwlock_unlock'
../../../src/.libs/libpmix.so: undefined reference to `pthread_join'
../../../src/.libs/libpmix.so: undefined reference to `pthread_create'
../../../src/.libs/libpmix.so: undefined reference to `pthread_mutexattr_init'
../../../src/.libs/libpmix.so: undefined reference to `pthread_mutexattr_destroy'
../../../src/.libs/libpmix.so: undefined reference to `pthread_rwlockattr_destroy'
../../../src/.libs/libpmix.so: undefined reference to `pthread_setaffinity_np'
collect2: error: ld returned 1 exit status
make[4]: *** [pevent] Erreur 1
make[4] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src/tools/pevent »
make[3]: *** [all-recursive] Erreur 1
make[3] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix/src »
make[2]: *** [all-recursive] Erreur 1
make[2] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/openpmix »
make[1]: *** [all-recursive] Erreur 1
make[1] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party »
make: *** [all-recursive] Erreur 1

Other Errors

make clean distclean
 ../configure --prefix=$WHERE_TO_INSTALL_OPENMPI_WITH_GCC --enable-debug --with-pmix=internal --with-hwloc=internal
`make VERBOSE=1 CFLAGS="-std=c99 -D_BSD_SOURCE"`

(Idem with -std=gnu99)

In file included from /root/openmpi-5.0.0/3rd-party/prrte/src/mca/oob/tcp/oob_tcp_component.h:38:0,
                 from ../../../../../../../3rd-party/prrte/src/mca/oob/tcp/oob_tcp_common.c:69:
/root/openmpi-5.0.0/3rd-party/prrte/src/include/prte_stdatomic.h:26:31: erreur fatale: stdatomic.h : Aucun fichier ou dossier de ce type
 #        include <stdatomic.h>
                               ^
compilation terminée.
In file included from /root/openmpi-5.0.0/3rd-party/prrte/src/mca/oob/tcp/oob_tcp_component.h:38:0,
                 from ../../../../../../../3rd-party/prrte/src/mca/oob/tcp/oob_tcp_listener.c:73:
/root/openmpi-5.0.0/3rd-party/prrte/src/include/prte_stdatomic.h:26:31: erreur fatale: stdatomic.h : Aucun fichier ou dossier de ce type
 #        include <stdatomic.h>
                               ^
compilation terminée.
In file included from /root/openmpi-5.0.0/3rd-party/prrte/src/mca/oob/tcp/oob_tcp_component.h:38:0,
                 from ../../../../../../../3rd-party/prrte/src/mca/oob/tcp/oob_tcp.c:68:
/root/openmpi-5.0.0/3rd-party/prrte/src/include/prte_stdatomic.h:26:31: erreur fatale: stdatomic.h : Aucun fichier ou dossier de ce type
 #        include <stdatomic.h>
                               ^
compilation terminée.
In file included from /root/openmpi-5.0.0/3rd-party/prrte/src/mca/oob/tcp/oob_tcp_component.h:38:0,
                 from ../../../../../../../3rd-party/prrte/src/mca/oob/tcp/oob_tcp_component.c:93:
/root/openmpi-5.0.0/3rd-party/prrte/src/include/prte_stdatomic.h:26:31: erreur fatale: stdatomic.h : Aucun fichier ou dossier de ce type
 #        include <stdatomic.h>
                               ^
compilation terminée.
make[4]: *** [oob_tcp_common.lo] Erreur 1
make[4]: *** Attente des tâches non terminées....
make[4]: *** [oob_tcp_listener.lo] Erreur 1
make[4]: *** [oob_tcp.lo] Erreur 1
make[4]: *** [oob_tcp_component.lo] Erreur 1
make[4] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/prrte/src/mca/oob/tcp »
make[3]: *** [all-recursive] Erreur 1
make[3] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/prrte/src »
make[2]: *** [all-recursive] Erreur 1
make[2] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party/prrte »
make[1]: *** [all-recursive] Erreur 1
make[1] : on quitte le répertoire « /root/openmpi-5.0.0/install_pmix_hwloc_internal_gcc_10_2_0/3rd-party »
make: *** [all-recursive] Erreur 1

Errors logs

make clean distclean
 ../configure --prefix=$WHERE_TO_INSTALL_OPENMPI_WITH_GCC --enable-debug --with-pmix=internal --with-hwloc=internal
`make VERBOSE=1 CFLAGS="-std=c99 -D_BSD_SOURCE"`

config.status.tar.gz config.lt.tar.gz config.log.tar.gz

[rhc54]

Not entirely sure I understand some of these reported problems. For one, I believe we require at least C99 nowadays, if not something even more modern - so I don't understand reporting errors for something older than that? I would have thought OMPI has a configure check to error out if you try it - yes? I wonder if you are bypassing that somehow.

I'm also not sure how the assert is generating an error in a non-debug build - isn't the assert macro supposed to translate to invisibility for non-debug builds? 🤷‍♂️ Easy enough to add a protection around it.

Would have to look at some of the others about PRRTE and PMIx - not seeing those from elsewhere, so must be something compiler/platform specific.

[rhc54]

Something seems very odd here. We build on a wide range of systems and compilers, can have never had to include such flags - nor seen any of these reported errors. Is this some local-built compiler, perhaps built with certain flags set? Can you tell us something more about this environment?

Frankly, I'm having a hard time making sense of the reported errors.

[jsquyres]

@W-Wuxian I think that this has been fixed in v5.0.1. Can you confirm that it now works for you?

[github-actions[bot]]

It looks like this issue is expecting a response, but hasn't gotten one yet. If there are no responses in the next 2 weeks, we'll assume that the issue has been abandoned and will close it.

[github-actions[bot]]

Per the above comment, it has been a month with no reply on this issue. It looks like this issue has been abandoned.

I'm going to close this issue. If I'm wrong and this issue is not abandoned, please feel free to re-open it. Thank you!