Compilation fails when compiling opencv_hdf with MPI

[AndreiCostinescu]

System Information

OpenCV version: 4.7.0 (also tested with 4.10.0) Operating System: Ubuntu 22.04 (also tested with 24.04) Compiler version: GCC 11.4.0 (also tested with 13.3.0) CMake version: 3.22 (also tested with 3.28)

Detailed description

Dear OpenCV community,

When compiling opencv with opencv_contrib with the cmake flag -DHDF5_PREFER_PARALLEL=ON and the compilation reaches the opencv_hdf module, the compilation fails with the following error:

In file included from /usr/include/hdf5/openmpi/hdf5.h:22,
                 from /usr/local/src/opencv_contrib/modules/hdf/src/hdf5.cpp:37:
/usr/include/hdf5/openmpi/H5public.h:63:13: fatal error: mpi.h: No such file or directory
   63 | #   include <mpi.h>
      |             ^~~~~~~

When changing the cmake flag to -DHDF5_PREFER_PARALLEL=OFF, then the compilation of the opencv_hdf module succeeds. libopenmpi-dev is installed on my system and I successfully compiled a test program on my system that includes the mpi.h file located on my system at /usr/lib/x86_64-linux-gnu/openmpi/include (albeit with the mpic++ script, not simply the gcc/g++ compiler), so the issue does not seem to lie with the file not existing on my system. I also tried to edit the CMakeLists.txt file in opencv_contrib/modules/hdf/CMakeLists.txt and add the directory where mpi.h lies to the HDF5_INCLUDE_DIRS variable, but this also did not help with the compilation; still the same error as above.

How should I solve this issue? 😄

Steps to reproduce

wget https://github.com/opencv/opencv/archive/refs/tags/4.10.0.tar.gz && \
  tar -xf 4.10.0.tar.gz && \
  rm 4.10.0.tar.gz && \
  mv opencv-4.10.0 opencv && \
  wget https://github.com/opencv/opencv_contrib/archive/refs/tags/4.10.0.tar.gz && \
  tar -xf 4.10.0.tar.gz && \
  rm 4.10.0.tar.gz && \
  mv opencv_contrib-4.10.0 opencv_contrib && \
  cd opencv && \
  mkdir build-release && \
  cd build-release && \
  cmake .. -DCMAKE_BUILD_TYPE=Release -DHDF5_PREFER_PARELLEL=ON -DOPENCV_EXTRA_MODULES_PATH="../../opencv_contrib/modules" && \
  make -j6 opencv_hdf

Issue submission checklist

• [X] I report the issue, it's not a question
• [X] I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
• [X] I updated to the latest OpenCV version and the issue is still there
• [x] There is reproducer code and related data files (videos, images, onnx, etc)

[Kumataro]

I have a slight recollection that I investigated the likely problem before. I wll investigate this problem this weekend more. This is only memo, sorry.

HDF module uses find_package(). So usually cmake should be set correct libraries and include dir information in envieonment variables.

https://github.com/opencv/opencv_contrib/blob/3e776c876ef8527ad639d2c792521b6cede291e5/modules/hdf/CMakeLists.txt#L19-L21

  if(NOT CMAKE_CROSSCOMPILING) # iOS build should not reuse OSX package
    find_package(HDF5)
  endif()

https://cmake.org/cmake/help/latest/module/FindHDF5.html

ocv_target_link_libraries(${the_module} ${HDF5_LIBRARIES})
ocv_include_directories(${HDF5_INCLUDE_DIRS})

But there are some trouble with hdf5-openmpi or hdf5-mpich instead of hdf5-serial. I only remember using HDF5_IS_PARALLEL is needed, but I forget what I have to do...

[Kumataro]

I found old my work, but it seems not good solution

https://github.com/opencv/opencv_contrib/issues/3216 / https://github.com/opencv/opencv_contrib/issues/3447

I think new option to switch reference hdf5-serial hdf5-opempi or hdf5-mpich is required instead of using findHDF5 or adding extra libraries manually.

kmtr@kmtr-VMware-Virtual-Platform:~/work$ pkgconf --list-all | grep hdf
hdf5-serial                    HDF5 - Hierarchical Data Format 5 (HDF5) - serial version
hdf5-mpi                       HDF5 - Hierarchical Data Format 5 (HDF5) - openmpi version
hdf5-mpich                     HDF5 - Hierarchical Data Format 5 (HDF5) - mpich version
hdf5-openmpi                   HDF5 - Hierarchical Data Format 5 (HDF5) - openmpi version
hdf5                           HDF5 - Hierarchical Data Format 5 (HDF5) - openmpi version

kmtr@kmtr-VMware-Virtual-Platform:~/work$ pkgconf hdf5-serial --libs --cflags
-I/usr/include/hdf5/serial -L/usr/lib/x86_64-linux-gnu/hdf5/serial -lhdf5
kmtr@kmtr-VMware-Virtual-Platform:~/work$ pkgconf hdf5-mpich --libs --cflags
-I/usr/include/hdf5/mpich -I/usr/lib/x86_64-linux-gnu/mpich/include -L/usr/lib/x86_64-linux-gnu/hdf5/mpich -lhdf5 -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -Wl,-z,relro -L/usr/lib/x86_64-linux-gnu/mpich/lib -lmpich -lpthread -lhwloc -lpmix -lucp -lucs
kmtr@kmtr-VMware-Virtual-Platform:~/work$ pkgconf hdf5-mpi --libs --cflags
-I/usr/include/hdf5/openmpi -I/usr/lib/x86_64-linux-gnu/openmpi/include -I/usr/lib/x86_64-linux-gnu/fortran/gfortran-mod-15/openmpi -I/usr/lib/x86_64-linux-gnu/openmpi/include/openmpi -L/usr/lib/x86_64-linux-gnu/hdf5/openmpi -lhdf5 -L/usr/lib/x86_64-linux-gnu/openmpi/lib -lmpi

[Kumataro]

Thank you for waiting, I believe this problem will be fixed with my patch .

Memo: I tried using pkg-config to swtich Open MPI or MPICH. But linker reports missing depended ucp/ucx libraries in system folder, even if they are exists(it is only MPICH).

So I use find_package(MPI) as standard way. If you want to select mpi implemntation, `update-alternatives is helpful.