Unable to build with cmake: git submodule errors

[rubenww]

trafficstars

I am trying to build pFUnit, I've tried the following:

 $ git clone --recursive https://github.com/Goddard-Fortran-Ecosystem/pFUnit/
Cloning into 'pFUnit'...
remote: Enumerating objects: 9307, done.
remote: Counting objects: 100% (138/138), done.
remote: Compressing objects: 100% (105/105), done.
remote: Total 9307 (delta 53), reused 55 (delta 23), pack-reused 9169
Receiving objects: 100% (9307/9307), 12.09 MiB | 2.54 MiB/s, done.
Resolving deltas: 100% (6657/6657), done.
Checking out files: 100% (369/369), done.
Submodule 'fArgParse' (https://github.com/Goddard-Fortran-Ecosystem/fArgParse.git) registered for path 'extern/fArgParse'
Cloning into 'extern/fArgParse'...
remote: Enumerating objects: 710, done.
remote: Counting objects: 100% (100/100), done.
remote: Compressing objects: 100% (71/71), done.
remote: Total 710 (delta 38), reused 69 (delta 23), pack-reused 610
Receiving objects: 100% (710/710), 241.92 KiB | 0 bytes/s, done.
Resolving deltas: 100% (394/394), done.
Submodule path 'extern/fArgParse': checked out '920e4fc7938cffe43ca58982259081f8808eb043'
Submodule 'gFTL-shared' (https://github.com/Goddard-Fortran-Ecosystem/gFTL-shared.git) registered for path 'extern/gFTL-shared'
Cloning into 'extern/gFTL-shared'...
remote: Enumerating objects: 1079, done.
remote: Counting objects: 100% (344/344), done.
remote: Compressing objects: 100% (171/171), done.
remote: Total 1079 (delta 197), reused 284 (delta 157), pack-reused 735
Receiving objects: 100% (1079/1079), 167.08 KiB | 0 bytes/s, done.
Resolving deltas: 100% (758/758), done.
Submodule path 'extern/fArgParse/extern/gFTL-shared': checked out 'b0a2df1c14d4ddca3f3bd0c56f5e737b946f7c19'
Submodule 'gFTL' (https://github.com/Goddard-Fortran-Ecosystem/gFTL.git) registered for path 'extern/gFTL'
Cloning into 'extern/gFTL'...
remote: Enumerating objects: 2660, done.
remote: Counting objects: 100% (1636/1636), done.
remote: Compressing objects: 100% (732/732), done.
remote: Total 2660 (delta 1087), reused 1357 (delta 877), pack-reused 1024
Receiving objects: 100% (2660/2660), 563.65 KiB | 571.00 KiB/s, done.
Resolving deltas: 100% (1716/1716), done.
Submodule path 'extern/fArgParse/extern/gFTL-shared/extern/gFTL': checked out '34bbeddd9192332258f820ea2ae06772552d54e6'
$ cd pFUnit/
$ mkdir build
$ cd build
$ cmake .. -DSKIP_MPI=YES -DSKIP_OPENMP=YES
-- The Fortran compiler identification is Intel 16.0.2.20160204
-- The C compiler identification is GNU 4.8.5
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: /opt/intel/bin/ifort - skipped
-- Checking whether /opt/intel/bin/ifort supports Fortran 90
-- Checking whether /opt/intel/bin/ifort supports Fortran 90 - yes
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- *** Setting default install prefix to ~/pFUnit/build/installed.
-- *** Override with -DCMAKE_INSTALL_PREFIX=<path>.
-- Setting build type to 'Debug' as none was specified.
-- Found Python: /usr/bin/python3.6 (found version "3.6.8") found components: Interpreter
You need to run this command from the toplevel of the working tree.
CMake Error at cmake/build_submodule.cmake:29 (message):
  git submodule update --init failed with 1, please checkout submodules
Call Stack (most recent call first):
  src/funit/CMakeLists.txt:27 (build_submodule)
  src/funit/core/CMakeLists.txt:89 (funit_target_link_pfunit)


-- Configuring incomplete, errors occurred!
See also "~/pFUnit/build/CMakeFiles/CMakeOutput.log".

But the submodules are there. Any ideas what I should do to fix this?

[tclune]

Hmm. I think I encountered this recently as well. A side effect of some otherwise well-meaning changes to allow pFUnit to be built embedded within other applications.

Probably the easiest workaround if your environment supports it, is to clone without the --recursive option. pFUnit should then clone the submodules during the build process. But I'll try to replicate this problem as well, as I certainly want the build to work either way.

[rubenww]

Thank you! I found a workaround, the error is here:

https://github.com/Goddard-Fortran-Ecosystem/pFUnit/blob/93dfc7705a06fc5cbf6d38efbb425483cbbe0073/cmake/build_submodule.cmake#L25

And the same error is in one of the 'external' submodule files. I had to manually set the working directory, because it was not pointing to the toplevel directory.

[tclune]

OK - glad you found it. Is it possible that this should be ${CMAKE_CURRENT_LIST_DIR}? (asking @ZedThree)

[ZedThree]

I'm pretty sure this error is because you've got quite an old version of git that requires git submodule commands to be run from the top-level directory. If you're on a cluster, it's probably worth checking if there's a more recent version of git in the module system, as there's quite a difference in performance compared to v1.x.

I think the correct fix for pFUnit is: WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}. Please could you try that fix @rubenrivm ?

[rubenww]

Sorry for my late reply. Thanks @ZedThree, I can confirm that setting the working directory to ${PROJECT_SOURCE_DIR} fixes the problem for me (git version is indeed 1.8.3.1). I had to make the same change to extern/fArgParse/extern/gFTL-shared/cmake/build_submodule.cmake for it to work.

[Leonard-Reuter]

I encountered the same problem, also with an old version of git. I read the build_submodules definition in pFUnit and simply compiled with -DGIT_SUBMODULE=OFF. Surprisingly it worked and the submodule where still there. This is really odd, there seems to be some doubling with the initialization of the git submodules, if pFUnit is already cloned recursively. Does the build_submodule routine only exist, such that the user does not have to write "git clone --recursive .." ?

[tclune]

Does the build_submodule routine only exist, such that the user does not have to write "git clone --recursive .." ?

I forget the full history at this point, but it is possible/probably that this was my initial motivation. I wanted to make it as easy-as-possible for a new user to get started. Short of actually writing any documentation, that is. ;-) This came at the price of buggy logic in my cmake implementation.

I wish that I could just assume that users would not be put off by the extra dependencies and be expected to separately install gFTL, gFTL-shared, and fArgParse. Easy, but I can see haw that would be daunting to a new user, esp. if they don't know cmake. But it is how I generally build, which is partly why I'm not always aware of the other approaches breaking. The CI could probably be expanded to check all possible ways of building, but ... not my top priority at the moment.