Cross-compilation support

[musm]

Lack of cross-compilation from linux to other platforms would significantly help distribution of binaries in the Julia ecosystem. This has been a long-standing issue. See e.g. https://github.com/JuliaPackaging/Yggdrasil/pull/567, https://forum.hdfgroup.org/t/cross-compiling-for-windows/6735, to mention two threads. There have been several attempts at it including the fork https://github.com/stevengj/hdf5 and in addition Steven Varga has worked on looking at this in the past.

[musm]

any status or thoughts?

[gnuoyd]

On Mon, Feb 28, 2022 at 10:08:25AM -0800, Mustafa M wrote:

any status or thoughts?

HDF5 is on the path to easier cross-compilation, but I cannot forecast when the work will be completed. I have a couple of outstanding PRs that will help, PR #1400 and PR #1426.

Dave

[musm]

Thanks for the update, looks quite promising and glad to see progress

[simonbyrne]

Any updates on this? @gnuoyd I noticed both PRs have now been merged: what other pieces need to get done?

[byrnHDF]

#1410 also is related and I will document the CMake process there and then create a cross-compile document in the release_docs folder.

[giordano]

@byrnHDF trying to build https://github.com/HDFGroup/hdf5/commit/eac2cd54e209cfa9556174f3fc1a592533aa64ad (most recent commit in develop as of now), I still get plenty of TRY_RUNs:

[23:55:44] CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
[23:55:44]    TEST_LFS_WORKS_RUN (advanced)
[23:55:44] For details see /workspace/srcdir/hdf5/build/TryRunResults.cmake
[...]
[23:55:52] CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
[23:55:52]    H5_LDOUBLE_TO_LONG_SPECIAL_RUN (advanced)
[23:55:52]    H5_LDOUBLE_TO_LONG_SPECIAL_RUN__TRYRUN_OUTPUT (advanced)
[23:55:52] For details see /workspace/srcdir/hdf5/build/TryRunResults.cmake
[23:55:52] CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
[23:55:52]    H5_LONG_TO_LDOUBLE_SPECIAL_RUN (advanced)
[23:55:52]    H5_LONG_TO_LDOUBLE_SPECIAL_RUN__TRYRUN_OUTPUT (advanced)
[23:55:52] For details see /workspace/srcdir/hdf5/build/TryRunResults.cmake
[23:55:52] CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
[23:55:52]    H5_LDOUBLE_TO_LLONG_ACCURATE_RUN (advanced)
[23:55:52]    H5_LDOUBLE_TO_LLONG_ACCURATE_RUN__TRYRUN_OUTPUT (advanced)
[23:55:52] For details see /workspace/srcdir/hdf5/build/TryRunResults.cmake
[23:55:53] CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
[23:55:53]    H5_LLONG_TO_LDOUBLE_CORRECT_RUN (advanced)
[23:55:53]    H5_LLONG_TO_LDOUBLE_CORRECT_RUN__TRYRUN_OUTPUT (advanced)
[23:55:53] For details see /workspace/srcdir/hdf5/build/TryRunResults.cmake
[23:55:53] CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
[23:55:53]    H5_DISABLE_SOME_LDOUBLE_CONV_RUN (advanced)
[23:55:53]    H5_DISABLE_SOME_LDOUBLE_CONV_RUN__TRYRUN_OUTPUT (advanced)
[23:55:53] For details see /workspace/srcdir/hdf5/build/TryRunResults.cmake

See for example https://dev.azure.com/JuliaPackaging/Yggdrasil/_build/results?buildId=23874&view=logs&j=1da34be1-494f-551e-5051-6ec6842bd3b0&t=cb3052b7-7b92-5226-336b-5a172e41a1c3&l=242.

For reference, CMake invocation:

cmake .. -DCMAKE_INSTALL_PREFIX=${prefix} \
    -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} \
    -DHDF5_BUILD_CPP_LIB=OFF \
    -DONLY_SHARED_LIBS=ON \
    -DHDF5_BUILD_HL_LIB=ON \
    -DHDF5_ENABLE_Z_LIB_SUPPORT=ON \
    -DHDF5_ENABLE_SZIP_SUPPORT=OFF \
    -DHDF5_ENABLE_SZIP_ENCODING=OFF \
    -DBUILD_TESTING=OFF

Am I missing something?

[byrnHDF]

The next step is to set those try-run values in the your preset H5Tinit.c file. I think H5Tinit.c is created by the above. See src/CMakeLists.txt about line 1035. HDF5_USE_PREGEN needs to be set to TRUE. HDF5_USE_PREGEN_DIR needs to be where the pregenerated files are located.

References: https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html https://gitlab.com/embeddedlinux/libs/platform

[giordano]

I thought the entire point of supporting cross-compilation was to not have to manually set the output of try-run values for all possible target platforms.

[mkitti]

I think we just need the CMakeCache.txt files, no?

https://cmake.org/cmake/help/latest/command/try_run.html#behavior-when-cross-compiling

[mkitti]

Here's my src/build-MINGW64/CMakeCache.txt from a MINGW64 run: https://github.com/msys2/MINGW-packages/pull/15033

//Checking IF your system converts long double to (unsigned) long
// values with special algorithm
H5_LDOUBLE_TO_LONG_SPECIAL:INTERNAL=
//Result of TRY_COMPILE
H5_LDOUBLE_TO_LONG_SPECIAL_COMPILE:INTERNAL=TRUE
//Result of try_run()
H5_LDOUBLE_TO_LONG_SPECIAL_RUN:INTERNAL=1
//Checking IF correctly converting (unsigned) long long to long
// double values
H5_LLONG_TO_LDOUBLE_CORRECT:INTERNAL=1
//Result of TRY_COMPILE
H5_LLONG_TO_LDOUBLE_CORRECT_COMPILE:INTERNAL=TRUE
//Result of try_run()
H5_LLONG_TO_LDOUBLE_CORRECT_RUN:INTERNAL=0
//Checking IF your system can convert (unsigned) long to long double
// values with special algorithm
H5_LONG_TO_LDOUBLE_SPECIAL:INTERNAL=
//Result of TRY_COMPILE
H5_LONG_TO_LDOUBLE_SPECIAL_COMPILE:INTERNAL=TRUE
//Result of try_run()
H5_LONG_TO_LDOUBLE_SPECIAL_RUN:INTERNAL=1
...
//Checking IF the cpu is power9 and cannot correctly converting
// long double values
H5_DISABLE_SOME_LDOUBLE_CONV:INTERNAL=
//Result of TRY_COMPILE
H5_DISABLE_SOME_LDOUBLE_CONV_COMPILE:INTERNAL=TRUE
//Result of try_run()
H5_DISABLE_SOME_LDOUBLE_CONV_RUN:INTERNAL=1

[mkitti]

Sounds like we need to copy the values into the generated TryRunResults.cmake and then pass those settings into cmake -C TryRunResults.cmake per https://cmake.org/cmake/help/book/mastering-cmake/chapter/Cross%20Compiling%20With%20CMake.html#using-compile-checks

[byrnHDF]

Our latest develop has no errors compiling with mingw64 on an ubuntu 22.04 system. We do have an issue (not with try/run configuration) compiling with aarch64 on an ubuntu 22.04 system. The compile error: "Assumed value of MB_LEN_MAX wrong" along with a number of macro redefined [-Wmacro-redefined] warnings. the aarch64 path is "/opt/android-ndk-r25b-linux/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/bin/clang"

[byrnHDF]

Is your mingw64 on a windows machine? I haven't got one of those workers available.

[mkitti]

Yes, I'm running mingw64 under msys2 on windows.

[byrnHDF]

Maybe what is needed is some more preset values in Configure for mingw on windows environments? There is a section at the top that presets known values for windows platforms - around line 55.

[byrnHDF]

long double has always been trouble in cygwin, mingw on windows and such. dtarith test usually fails on those as well.

I think it has to do with compiler tricks. (Not official statement, just a personal opinion).

[byrnHDF]

ConversionTests.c file is an interesting read.

[mkitti]

We may be getting a bit distracted by the mingw-w64 aspect. As far as I can tell we have a solution to build on that platform. That is we have a functional native build solution for msys2/mingw-w64.

What I would like to do is send @giordano a file, my CMakeCache.txt for example, so that we can also build mingw-w64 binaries from Linux via cross compilation.

Cmake seems to have some very rudimentary infrastructure to support this. Basically, if we can transmit information cmake -C to populate the cmake cache, then cmake can just use that information.

The process seems to be as follows:

1. On the target platform, perform the cmake configuration step, which will populate a CMakeCache.txt. That CMakeCache.txt should contain all the values generated from the try_run and try_compile steps on the target platform. Rename this to CMakeCache-<target platform>.txt.
2. On the build platform, do an initial configuration build with cmake -DCMAKE_CROSSCOMPILING=ON. This will generate a TryRunResults.cmake file.
3. On the build platform, write and execute a script to move values from CMakeCache-<target platform>.txt to TryRunResults.cmake.
4. On the build platform, invoke cmake -C TryRunResults.cmake to populate the cmake cache.

[giordano]

TryRun isn't a great way to do cross-compilation though. Quite the opposite in fact. What https://github.com/stevengj/hdf5/commit/6b0545585282647bb8d7be8744d73745b000bdd3 showed is that most of those checks are actually compiler checks which don't need to run code in the first place.

[byrnHDF]

Yes that is basically the process. However, we have a mingw64.cmake toolchain file in the config/toolchain folder - have you used that?

[mkitti]

TryRun isn't a great way to do cross-compilation though. Quite the opposite in fact. What stevengj/hdf5@6b05455 showed is that most of those checks are actually compiler checks which don't need to run code in the first place.

If we wanted to bypass try_run and try_compile, we could use CMAKE_CROSSCOMPILING_EMULATOR to run autotools in order to return the result of the programs.

Per @derobins , The HDF Group is leaning towards retiring autotools in favor of cmake. I've commented there that an issue with this is it closing off Steven G. Johnson's route of populating this.

Perhaps the "emulation" route might be a way to salvage this?

[byrnHDF]

Eliminating tryrun has been discussed and is preferred. It is the details in doing it for all the platforms where hdf is used. I think that is where using the toolchain files can be part of the solution.

[mkitti]

Are you suggesting that we can the toolchain files located at the link below to include type information that try_run and try_compile are trying to obtain?

https://github.com/HDFGroup/hdf5/tree/develop/config/toolchain

[byrnHDF]

Maybe, I think it would be a good place for specific settings that are specific to that build env.

[byrnHDF]

Actually, a toolchain could be gotten from a URL using the FetchContent (CMake has an example)

[mkitti]

The Java CPP project has cross compiling setup:

https://github.com/bytedeco/javacpp-presets/tree/master/hdf5

[DoneListen]

Hello, has this problem been solved

[DoneListen]

I set these variables to OFF, and cmake succeeds. But there was a tricky problem when make： Generated files H5detect and H5make_libsettings, and report an error "H5make_libsettings: cannot execute binary file"； What is the current solution to this

[mkitti]

As far as I know this has not been done. Preferably, the need to run an executable on the target platform could be replaced by some static assertions in C and Fortran so that the settings could be detected by compilation alone.

https://en.cppreference.com/w/c/language/_Static_assert https://en.cppreference.com/w/cpp/language/static_assert

https://fortran-lang.discourse.group/t/static-assert-in-fortran/5687

[DoneListen]

H5make_libsettings

H5make_libsettings and H5detect is used to generate some files, I am not sure what you mean.