Compiling on Eagle with `OPENFAST_CPP_API` turned on

[rthedin]

Bug description Compilation of OpenFAST with Intel compilers and -DBUILD_OPENFAST_CPP_API=ON flag on Eagle fails. Several modules need to be added, but some of them aren't very obvious. I had the issue using v3.1.0 tag.

To Reproduce <Update the following list with your specific information.> Steps to reproduce the behavior:

module load comp-intel intel-mpi mkl cmake
git checkout v3.1.0
mkdir build
cd build
cmake .. \
  -DBUILD_FASTFARM=ON \
  -DBUILD_SHARED_LIBS=ON \
  -DCMAKE_Fortran_FLAGS_RELEASE="-O2 -xhost" \
  -DDOUBLE_PRECISION:BOOL=OFF \
  -DOPENMP=ON \
  -DCMAKE_BUILD_TYPE=Release \
  -DBUILD_OPENFAST_CPP_API=ON \
  -DCMAKE_Fortran_COMPILER=/nopt/nrel/apps/compilers/intel/2020.1.217/compilers_and_libraries_2020.1.217/linux/bin/intel64/ifort \
  -DCMAKE_C_COMPILER=/nopt/nrel/apps/compilers/intel/2020.1.217/compilers_and_libraries_2020.1.217/linux/bin/intel64/icc \
  -DCMAKE_CXX_COMPILER=/nopt/nrel/apps/compilers/intel/2020.1.217/compilers_and_libraries_2020.1.217/linux/bin/intel64/icpc

Expected behavior The call to cmake should be successful. In order for it to go through, additional modules should be loaded. The following works:

module purge
module use /nopt/nrel/apps/modules/centos77/modulefiles
module load comp-intel/2020.1.217
module load intel-mpi/2020.1.217
module load mkl/2020.1.217
module use /nopt/nrel/ecom/wind/spack/share/spack/modules/linux-centos7-x86_64/gcc-8.4.0
module load yaml-cpp-0.6.2
module load cmake
module load hdf5/1.12.0/intel-impi
module unload gcc

cmake .. -DBUILD_FASTFARM=ON -DBUILD_SHARED_LIBS=ON -DCMAKE_Fortran_FLAGS_RELEASE="-O2 -xhost" -DDOUBLE_PRECISION:BOOL=OFF -DOPENMP=ON -DCMAKE_BUILD_TYPE=Release -DBUILD_OPENFAST_CPP_API=ON -DCMAKE_Fortran_COMPILER=/nopt/nrel/apps/compilers/intel/2020.1.217/compilers_and_libraries_2020.1.217/linux/bin/intel64/ifort -DCMAKE_C_COMPILER=/nopt/nrel/apps/compilers/intel/2020.1.217/compilers_and_libraries_2020.1.217/linux/bin/intel64/icc -DCMAKE_CXX_COMPILER=/nopt/nrel/apps/compilers/intel/2020.1.217/compilers_and_libraries_2020.1.217/linux/bin/intel64/icpc

OpenFAST Version Github tag v3.1.0

System Information (please complete the following information):

• Using NREL's Eagle HPC system
• OS: CentoOS 7
• Compiler: forcing the Intel compilers as indiced by the cmake call. If not giving intel compilers that way, cmake always resorts to gcc.
• Compiler settings: cmake flags given in the call above

[rthedin]

@andrew-platt FYI

[rafmudaf]

@rthedin is this an issue with the OpenFAST CMake project or the Eagle environment?

Also, its generally safer to avoid modifying the build-type cmake flags variable (i.e. CMAKE_Fortran_FLAGS_RELEASE) and instead use CMAKE_Fortran_FLAGS. Also, in your cmake line, its not clear whether you're compiling with -O2 or -O3 since you set O2 explicitly but also set the build type to RELEASE which uses O3. I think make will use the one that is defined last. In your case, I suggest using CMAKE_BUILD_TYPE=RELWITHDEBINFO if you want to use O2 and don't set it directly. This will create a larger executable since it has the debug symbols but it will use O2. In your current configuration, you may be getting the correct behavior but this is more explicit.

Something like this...

cmake .. \
  -DBUILD_FASTFARM=ON \
  -DBUILD_SHARED_LIBS=ON \
  -DCMAKE_Fortran_FLAGS="-xhost" \
  -DDOUBLE_PRECISION:BOOL=OFF \
  -DOPENMP=ON \
  -DCMAKE_BUILD_TYPE=RelWithDebInfo \
  -DBUILD_OPENFAST_CPP_API=ON \
  -DCMAKE_Fortran_COMPILER=/nopt/nrel/apps/compilers/intel/2020.1.217/compilers_and_libraries_2020.1.217/linux/bin/intel64/ifort \
  -DCMAKE_C_COMPILER=/nopt/nrel/apps/compilers/intel/2020.1.217/compilers_and_libraries_2020.1.217/linux/bin/intel64/icc \
  -DCMAKE_CXX_COMPILER=/nopt/nrel/apps/compilers/intel/2020.1.217/compilers_and_libraries_2020.1.217/linux/bin/intel64/icpc

[andrew-platt]

As I recall the O2 was a recommendation from a few years ago when there were numerical differences between O2 and O3 compilations, and before we set the FAST_Types.f90 and FAST_Farm_Types.f90 to use O2.

I think I had given @kshaler this recommendation a few years ago. So this is legacy carryover from her setup scripts.

[rthedin]

There is actually no issues with either the CMake or the Eagle environment. Andy helped me with some issues I ran into while compiling it and he suggested I submit an issue. The goal of this issue is really just to highlight the need for better documentation when compiling from source using different compilers. I used some flags from this document (section 2.2.2) and added some other based on the current readthedocs. That document is what Andy is referring to legacy carryover.

I got stuck a few times until I added all the necessary modules. They are the ones I listed in the original message above. I think it could be useful to have those modules listed in the compilation part of the documentation.

[andrew-platt]

Thanks for posting @rthedin! I mostly wanted to make sure we had this info someplace so that we can get it into documentation (maybe we should convert this to a discussion?).