Incomplete library exports on Windows

[letmaik]

Environment Information

• What platform are you using? (please provide specific distribution/version in summary)
  • [ ] Linux
  • [x] Windows
  • [ ] OSX
  • [ ] Other
  • [ ] NA
• 32 and/or 64 bit?
  • [ ] 32-bit
  • [x] 64-bit
• What build system are you using?
  • [ ] autotools (configure)
  • [x] cmake

Summary of Issue

The pre-built netcdf-c libraries at http://www.unidata.ucar.edu/software/netcdf/docs/winbin.html are missing certain DLL symbols like nc_create_par_fortran, nc_open_par_fortran, nc_var_par_access. This can be verified with the following command: dumpbin /exports libnetcdf.dll.

I also built netcdf-c 4.5.0 in an MSYS/mingw64 environment using cmake and faced the same problem.

Not having those functions exported means that it is impossible to link netcdf-fortran against it as it requires those.

I couldn't exactly figure out yet what the reason is but I noticed that there are two files which contain overlapping function declarations with different export keywords: include/netcdf_par.h (not included without parallel option) and include/netcdf_f.h (always included). Maybe that gives a clue?

[WardF]

Thank you for reporting; I will get this fixed!

[edhartnett]

Those functions are only used for parallel I/O builds of netCDF. Did you intend to use parallel IO on Windows? That is not typical, as far as I know.

So it seems like the windows DLL should not include those functions.

[letmaik]

Thanks for looking at it.

No I didn't want to use parallel IO but it seems that netcdf-fortran has a hard dependency on those functions, see https://github.com/Unidata/netcdf-fortran/blob/78a9839366cf8cdc929da450ebba9fad941c3341/fortran/nf_nc4.f90#L61. This file is included whenever USE_NETCDF4 is enabled, see https://github.com/Unidata/netcdf-fortran/blob/78a9839366cf8cdc929da450ebba9fad941c3341/fortran/Makefile.am#L58.

From what I can see, netcdf_h.h is included unconditionally in libdispatch in dvar.c which is always built. https://github.com/Unidata/netcdf-c/blob/327c19499cc37f4d04cdab728f116c95a87935f8/libdispatch/dvar.c#L8-L9 Shouldn't that then mean that those functions are always included irrelevant of the parallel IO build flag?

[WardF]

We currently don't officially support Fortran on Windows, which is why this hasn't received as much attention. But I should be able to address that.

[WardF]

I assume you are using mingw64/fortran equivalent and not Visual Studio or some such via the MSYS shell?

[letmaik]

I use the mingw64 shell from the MSYS2 package. Note that the person from the linked issue used the Intel Fortran compiler on Windows and had the same problem. Make sure to build netcdf-fortran as shared library, otherwise you may not notice the undefined symbols.

[WardF]

Part of the problem is a lack of access to free fortran compilers compatible with visual studio; I understand that PGI has released a community edition of their compilers, so when I get a chance I will try to get it working with that.

[letmaik]

OK great, I didn't know that about the PGI compiler. On their feature comparison page it says that the free version doesn't have Visual Studio integration. Feel free to contact me via email if I can be of any help with reproducing it or setting up mingw64. I'm currently a bit blocked by this issue and would really like to get this solved. Once fixed, netcdf-fortran could be added to the msys/mingw package repository which would make it a lot easier to build scientific software on Windows that uses netcdf-fortran.

[dmey]

Hi @WardF, I have encountered the same issue with the installation of netcdf-fortran as reported above. Could you please let me know if this issue is currently being actively looked at? If not, is this something that you may be able to shed some light on? -- Perhaps we can try to address the issue collaboratively? Thank you.

[WardF]

Thanks, I have mingw64 set up and will gladly take a look at this as soon as I can; it is definitely being looked at, but there are a number of issues and pull requests currently being reviewed as well. My assumption is that the functions just need to be exported as other functions have needed to be on (on Windows). @letmaik just for the sake of completion, can you share how you are building netcdf-c on Windows with cmake using MSYS2/mingw64? I work in MSYS2 already, so getting it set up won't be a problem.

[dmey]

Thanks @WardF, if it is of any help, these are the installation steps I followed:

1. Open the MSYS2 MSYS command prompt and install dependencies using the pacman package manager

pacman -S mingw-w64-x86_64-toolchain
pacman -S mingw-w64-x86_64-netcdf

2. Open the the MSYS2 MinGW 64-bit command prompt to configure netcdf-fortran with CMake

cd /tmp
wget ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-fortran-4.4.4.tar.gz
tar xvzf netcdf-fortran-4.4.4.tar.gz
cd netcdf-fortran-4.4.4
mkdir build && cd build
cmake -G "MSYS Makefiles" -DCMAKE_BUILD_TYPE=Release -DENABLE_TESTS=OFF ..

which produces the following output:

-- The Fortran compiler identification is GNU 7.2.0
-- The C compiler identification is GNU 7.2.0
-- Check for working Fortran compiler: C:/msys64/mingw64/bin/gfortran.exe
-- Check for working Fortran compiler: C:/msys64/mingw64/bin/gfortran.exe  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether C:/msys64/mingw64/bin/gfortran.exe supports Fortran 90
-- Checking whether C:/msys64/mingw64/bin/gfortran.exe supports Fortran 90 -- yes
-- Check for working C compiler: C:/msys64/mingw64/bin/gcc.exe
-- Check for working C compiler: C:/msys64/mingw64/bin/gcc.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Looking for nc_def_opaque in netcdf
-- Looking for nc_def_opaque in netcdf - not found
-- Looking for nccreate in netcdf
-- Looking for nccreate in netcdf - not found
-- Looking for nc_set_log_level in netcdf
-- Looking for nc_set_log_level in netcdf - not found
-- Looking for oc_open in netcdf
-- Looking for oc_open in netcdf - not found
-- Looking for nc_use_parallel_enabled in netcdf
-- Looking for nc_use_parallel_enabled in netcdf - not found


-- CMake Summary:

--      Build Type:                     Release
--      Building Shared Libraries:      ON
--      Building netCDF-4:
--      Building DAP Support:
--      Logging Support:                OFF
--      Using Parallel IO:              OFF
--      Linking against:                netcdf;

-- Configuring done
-- Generating done
-- Build files have been written to: C:/msys64/tmp/netcdf-fortran-4.4.4/build

3. Install

make VERBOSE=1

which fails with the following output:

--- tail ---
[ 41%] Linking Fortran shared library libnetcdff.dll
cd /C/msys64/tmp/netcdf-fortran-4.4.4/build/fortran && /C/msys64/mingw64/bin/cmake.exe -E remove -f CMakeFiles/netcdff.dir/objects.a
cd /C/msys64/tmp/netcdf-fortran-4.4.4/build/fortran && /C/msys64/mingw64/bin/ar.exe cr CMakeFiles/netcdff.dir/objects.a "CMakeFiles/netcdff.dir/typeSizes.f90.obj" "CMakeFiles/netcdff.dir/module_netcdf_nc_data.F90.obj" "CMakeFiles/netcdff.dir/module_netcdf_nc_interfaces.f90.obj" "CMakeFiles/netcdff.dir/module_netcdf_nf_data.F90.obj" "CMakeFiles/netcdff.dir/module_netcdf_nf_interfaces.F90.obj" "CMakeFiles/netcdff.dir/module_netcdf_f03.f90.obj" "CMakeFiles/netcdff.dir/nf_attio.F90.obj" "CMakeFiles/netcdff.dir/nf_control.F90.obj" "CMakeFiles/netcdff.dir/nf_dim.f90.obj" "CMakeFiles/netcdff.dir/nf_misc.f90.obj" "CMakeFiles/netcdff.dir/nf_genatt.f90.obj" "CMakeFiles/netcdff.dir/nf_geninq.f90.obj" "CMakeFiles/netcdff.dir/nf_genvar.f90.obj" "CMakeFiles/netcdff.dir/nf_vario.F90.obj" "CMakeFiles/netcdff.dir/nf_var1io.F90.obj" "CMakeFiles/netcdff.dir/nf_varaio.F90.obj" "CMakeFiles/netcdff.dir/nf_varmio.F90.obj" "CMakeFiles/netcdff.dir/nf_varsio.F90.obj" "CMakeFiles/netcdff.dir/module_netcdf4_nc_interfaces.f90.obj" "CMakeFiles/netcdff.dir/module_netcdf4_nf_interfaces.F90.obj" "CMakeFiles/netcdff.dir/module_netcdf4_f03.f90.obj" "CMakeFiles/netcdff.dir/nf_lib.c.obj" "CMakeFiles/netcdff.dir/nf_nc4.f90.obj" "CMakeFiles/netcdff.dir/netcdf4.f90.obj" "CMakeFiles/netcdff.dir/nf_v2compat.c.obj" "CMakeFiles/netcdff.dir/module_netcdf_fortv2_c_interfaces.f90.obj" "CMakeFiles/netcdff.dir/nf_fortv2.f90.obj"
cd /C/msys64/tmp/netcdf-fortran-4.4.4/build/fortran && /C/msys64/mingw64/bin/gfortran.exe  -O2 -g -DNDEBUG  -shared -o libnetcdff.dll -Wl,--out-implib,libnetcdff.dll.a -Wl,--major-image-version,6,--minor-image-version,1 -Wl,--whole-archive CMakeFiles/netcdff.dir/objects.a -Wl,--no-whole-archive /C/msys64/mingw64/lib/libnetcdf.dll.a /mingw64/lib/libhdf5_hl.dll.a /mingw64/lib/libhdf5.dll.a /mingw64/lib/libz.dll.a -lgcc_eh -lgcc_eh
CMakeFiles/netcdff.dir/objects.a(nf_nc4.f90.obj): In function `nf_create_par_':
C:/msys64/tmp/netcdf-fortran-4.4.4/fortran/nf_nc4.f90:61: undefined reference to `nc_create_par_fortran'
CMakeFiles/netcdff.dir/objects.a(nf_nc4.f90.obj): In function `nf_open_par_':
C:/msys64/tmp/netcdf-fortran-4.4.4/fortran/nf_nc4.f90:94: undefined reference to `nc_open_par_fortran'
CMakeFiles/netcdff.dir/objects.a(nf_nc4.f90.obj): In function `nf_var_par_access_':
C:/msys64/tmp/netcdf-fortran-4.4.4/fortran/nf_nc4.f90:121: undefined reference to `nc_var_par_access'
collect2.exe: error: ld returned 1 exit status
make[2]: *** [fortran/CMakeFiles/netcdff.dir/build.make:779: fortran/libnetcdff.dll] Error 1
make[2]: Leaving directory '/tmp/netcdf-fortran-4.4.4/build'
make[1]: *** [CMakeFiles/Makefile2:983: fortran/CMakeFiles/netcdff.dir/all] Error 2
make[1]: Leaving directory '/tmp/netcdf-fortran-4.4.4/build'
make: *** [Makefile:141: all] Error 2

[letmaik]

Here are my steps:

• Download/Install 64bit MSYS2 from http://repo.msys2.org/distrib/x86_64/msys2-x86_64-20161025.exe
• Inside the MSYS shell:
  • pacman -Syu (system upgrade)
  • Forcefully close and reopen MSYS shell
  • pacman -Syu (again, to update packages)
  • pacman -S mingw-w64-x86_64-toolchain mingw-w64-x86_64-cmake mingw-w64-x86_64-libtool make tar mingw-w64-x86_64-hdf5
• Inside the MinGW 64-bit shell (since we want to build fully native Windows libraries):
  • wget ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-4.5.0.tar.gz
  • tar xvzf netcdf-4.5.0.tar.gz
  • cd netcdf-4.5.0
  • ./configure --enable-dll --disable-dynamic-loading --disable-examples --disable-utilities
  • make && make install
  • cd ..
  • wget ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-fortran-4.4.4.tar.gz
  • tar xvzf netcdf-fortran-4.4.4.tar.gz
  • cd netcdf-fortran-4.4.4
  • Using configure I couldn't get it to generate a shared library, so I used CMake:
  • mkdir build && cd build
  • cmake -G "MSYS Makefiles" ..
  • make

Output of cmake -G "MSYS Makefiles" ..:

-- The Fortran compiler identification is GNU 7.2.0
-- The C compiler identification is GNU 7.2.0
-- Check for working Fortran compiler: C:/msys64/mingw64/bin/gfortran.exe
-- Check for working Fortran compiler: C:/msys64/mingw64/bin/gfortran.exe  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether C:/msys64/mingw64/bin/gfortran.exe supports Fortran 90
-- Checking whether C:/msys64/mingw64/bin/gfortran.exe supports Fortran 90 -- yes
-- Check for working C compiler: C:/msys64/mingw64/bin/gcc.exe
-- Check for working C compiler: C:/msys64/mingw64/bin/gcc.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found Doxygen: C:/Program Files/doxygen/bin/doxygen.exe (found version "1.8.12") found components:  doxygen missing components:  dot
-- Found netcdf: C:/msys64/mingw64/lib/libnetcdf.dll.a
-- Looking for nc_def_opaque in C:/msys64/mingw64/lib/libnetcdf.dll.a
-- Looking for nc_def_opaque in C:/msys64/mingw64/lib/libnetcdf.dll.a - found
-- Looking for nccreate in C:/msys64/mingw64/lib/libnetcdf.dll.a
-- Looking for nccreate in C:/msys64/mingw64/lib/libnetcdf.dll.a - found
-- Looking for nc_set_log_level in C:/msys64/mingw64/lib/libnetcdf.dll.a
-- Looking for nc_set_log_level in C:/msys64/mingw64/lib/libnetcdf.dll.a - not found
-- Looking for oc_open in C:/msys64/mingw64/lib/libnetcdf.dll.a
-- Looking for oc_open in C:/msys64/mingw64/lib/libnetcdf.dll.a - not found
-- Looking for nc_use_parallel_enabled in C:/msys64/mingw64/lib/libnetcdf.dll.a
-- Looking for nc_use_parallel_enabled in C:/msys64/mingw64/lib/libnetcdf.dll.a - not found


-- CMake Summary:

--      Build Type:                     DEBUG
--      Building Shared Libraries:      ON
--      Building netCDF-4:              ON
--      Building DAP Support:
--      Logging Support:                OFF
--      Using Parallel IO:              OFF
--      Linking against:                C:/msys64/mingw64/lib/libnetcdf.dll.a;

Error running make:

[ 12%] Linking Fortran shared library libnetcdff.dll
CMakeFiles/netcdff.dir/objects.a(nf_nc4.f90.obj): In function `nf_create_par_':
C:/msys64/home/.../netcdf-fortran-4.4.4/fortran/nf_nc4.f90:61: undefined reference to `nc_create_par_fortran'
CMakeFiles/netcdff.dir/objects.a(nf_nc4.f90.obj): In function `nf_open_par_':
C:/msys64/home/.../netcdf-fortran-4.4.4/fortran/nf_nc4.f90:94: undefined reference to `nc_open_par_fortran'
CMakeFiles/netcdff.dir/objects.a(nf_nc4.f90.obj): In function `nf_var_par_access_':
C:/msys64/home/.../netcdf-fortran-4.4.4/fortran/nf_nc4.f90:121: undefined reference to `nc_var_par_access'
collect2.exe: error: ld returned 1 exit status
make[2]: *** [fortran/CMakeFiles/netcdff.dir/build.make:776: fortran/libnetcdff.dll] Error 1
make[1]: *** [CMakeFiles/Makefile2:983: fortran/CMakeFiles/netcdff.dir/all] Error 2
make: *** [Makefile:141: all] Error 2

[letmaik]

Some more details on why this happens on Windows and not on Linux:

• Windows only exports functions in a shared library if they have the correct export markers defined in some header file while compiling code. On Linux, all symbols are always exported.
• The concrete problem here is that functions in https://github.com/Unidata/netcdf-c/blob/master/libdispatch/dparallel.c#L88 are included within the build, but the corresponding header file which would define the necessary export declarations is NOT included from this file.
• Given that each function dparallel.c has a conditional empty implementation:

#ifndef USE_PARALLEL
   return NC_ENOPAR;
#else

... I assume that those functions are supposed to be available at link-time for external libraries at all times. And hence it is clearly a bug in netcdf-c and not netcdf-fortran, but do correct me if I'm wrong.

[WardF]

I believe you are correct. I will try to have this fixed shortly it should be pretty easy :)

[edhartnett]

I was not aware that anyone was using the enable-dll configure option, or that it worked at all!

Sadly I recently removed this option. I guess I better put it back!

So that is a pleasant surprise. I will add mingw to my CI system and restore the DLL stuff I removed from the automake files.

[edhartnett]

OK, I have restored the DLL sections to configure.ac and to the relevant Makefile.am files.

I have also change netcdf_par.h so that, like netcdf.h, the extra microsoft external declarations are present for every function. @letmaik Can you check to see if this solves your problems building fortran on windows? (You must build my branch from source to check, I can't build DLLs at the moment.)

Clone: [email protected]:NetCDF-World-Domination-Council/netcdf-c.git

Switch to branch ejh_fill_values and try it.

[letmaik]

@edhartnett I just tried it but get the same errors still. I saw you modified include/netcdf_par.h but this file is not included from dparallel.c which is where the implementation resides, so effectively your change doesn't have any effect on compilation. That's the main problem. Note that you can't directly include netcdf_par.h from dparallel.h as it uses MPI types which are not defined if USE_PARALLEL is not defined which would be the case here.

[edhartnett]

OK, thanks for checking. I have undone the attempted fix. I will have to slowly back away from this now...

[WardF]

@edhartnett No worries, I have been working on the fix and will get it in when I'm back in the office; this is similar to issues we've seen in the past with some other exports on Windows. Thanks!

[letmaik]

@WardF How are you progressing on this one? If you need someone for testing just let me know!

[letmaik]

It's been a while, any updates on this?

[WardF]

Hi, thanks for bumping this; let me go take a quick look and follow up :)

[WardF]

I've pushed a branch, gh554.wif, can you try that branch and see if the issue persists?

[letmaik]

Please see my earlier comment https://github.com/Unidata/netcdf-c/issues/554#issuecomment-354823851. You essentially tried the same here.

It would be good if you get a local setup to test it yourself, otherwise it takes too long to iterate on it I think.

[WardF]

I'm refreshing my memory of this issue, let me see if I can duplicate it on my side now.

[WardF]

Thinking out loud:

Ok, so it looks like some parallel functions are being defined compiled in (dparallel.c) regardless of whether USE_PARALLEL4 is defined. This seems to make sense, as there are no downstream checks in netcdf-fortran, it is just assumed that these functions are present. HOWEVER the functions aren't being exported unless USE_PARALLEL4 (or some such) is defined, hence the issue we are seeing reported here.

Fenceposting the functions behind USE_PARALLEL4 will require some downstream changes as well; for now a fix should incorporate the existing logic, these functions will be available and exposed whether parallel I/O is actually there or not. Therefor, we just need to make sure that when netcdf_par.h isn't being included in ncdispatch.h, we include netcdf_f.h instead.

Pursuing this line of reasoning, twe make it further but we now get a whole lot of other undefined references.

Also, it appears that current netcdf-c master won't build in this environment, so that needs untangled as well; I've been working with 4.6.1.

So, we have a fix for the immediate issue reported; the rest will have to be sorted out before we can offer official Windows support for netCDF-Fortran.

[DennisHeimbigner]

Would it be simpler to always export those functions but gut them when parallel is disabled?

[WardF]

Always exporting is the decision that was made previously, although I don't have any information regarding this decision. But we're past that issue and now seeing a new one that I'll document here. In current master, dap2/ does not build using the steps outlined above. I'll document the compiler errors here in a bit, at a glance I don't think it will be a difficult issue to solve. It is a curious state of affairs, however, as these are not issues we see on other platforms.

[scivision]

This is still an issue with MSYS2, same symptoms.