duplicate symbol typeinfo under MacOS

[hivon]

This problem looks a bit like #677. Under MacOS 10.14.6, with GNU CC 8.3.0 (as well as 9.2.0), the release 1.0.0-rc.2 (as well as the git version of March 24),
crashes at linking with the error message

[100%] Linking CXX executable gdl
duplicate symbol typeinfo name for Data_<SpDComplex> in:
    CMakeFiles/gdl.dir/datatypes.cpp.o
    CMakeFiles/gdl.dir/basic_op.cpp.o
duplicate symbol typeinfo for Data_<SpDComplex> in:
    CMakeFiles/gdl.dir/datatypes.cpp.o
    CMakeFiles/gdl.dir/basic_op.cpp.o
duplicate symbol typeinfo name for Data_<SpDUInt> in:
    CMakeFiles/gdl.dir/datatypes.cpp.o
    CMakeFiles/gdl.dir/basic_op.cpp.o
...
duplicate symbol typeinfo for Data_<SpDLong64> in:
    CMakeFiles/gdl.dir/datatypes.cpp.o
    CMakeFiles/gdl.dir/ofmt.cpp.o
ld: 252 duplicate symbols for architecture x86_64

Any way out ?

[alaingdl]

Unfortunately unable to replicate because the Mac I use with 10.14 is crashed at the observatory, physical access denied because of Covid :((

• I have an issue on OSX 10.12, to be solve (push) in "saverestore.cpp" (string issue) Apple LLVM version 8.0.0 (clang-800.0.42.1)

• no message when compiling on Debian 10 with Clang-7 clang version 7.0.1-8 (tags/RELEASE_701/final) CC=clang CXX=clang++ cmake .. -DPYTHON=OFF

[GillesDuvert]

can you confirm that the file instantiate_templates.cpp is as in https://github.com/gnudatalanguage/gdl/commit/144ab6e608ec886a80dcd64cc8be3c6dc53c3fb8#commitcomment-37668301 ?

[alaingdl]

The machine with OSX 10.14.4 was just restarted, compilation with Clang 10 is OK. I will test now GCC.

[[email protected]:]$ CC=clang CXX=clang++ cmake .. -DPYTHON=OFF -DOPENMP=OFF  \
-DGEOTIFF=OFF  -DSHAPELIB=OFF

[...]

[[email protected]:]$ clang --version
Apple LLVM version 10.0.0 (clang-1000.10.44.4)
Target: x86_64-apple-darwin18.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

[[email protected]:]$ sw_vers 
ProductName:	Mac OS X
ProductVersion:	10.14.4
BuildVersion:	18E226

[hivon]

Hi @GillesDuvert, I tried after adding the #ifdef lines in instantiate_templates.hpp (not .cpp which does not exist), but it makes no difference on the linking, as already noted by @opoplawski.

[alaingdl]

Replicated on OSX 10.14.4 with GCC 8, lot of messages (similar)

[[email protected]:]$  CC=gcc-8 CXX=g++-8 cmake .. -DPYTHON=OFF \
 -DOPENMP=OFF  -DGEOTIFF=OFF  -DSHAPELIB=OFF \
 -DWXWIDGETS=OFF -DHDF=OFF -DPSLIB=OFF -DEDITLINE=off -DMAGICK=off

unclear for me why so much dependencies should be switch OFF for GCC and not for Clang

[GillesDuvert]

This comes probably from the #include "datatypes.hpp" //for friend declaration that are still present in:

assocdata.cpp
basic_op.cpp
basic_op_add.cpp
basic_op_div.cpp
basic_op_mult.cpp
basic_op_new.cpp
basic_op_sub.cpp
basic_pro.cpp
default_io.cpp
ofmt.cpp

could you comment out the corresponding #include line in each of these files and check that re-compilation goes well? (it does, for me). All this is a bit murky.

[hivon]

Hi @GillesDuvert, I tried commenting out the #include "datatypes.hpp" lines in those files, but it does not help. @alaingdl: when I try to use clang instead of genuine gcc, I get a problem with graphicsmagick. Eric

[alaingdl]

same for me : if commented, compilation OK with Clang, failed with GCC

@hivon: same than me :( cmake ..... -Dmagick=off just for tests ?!

[alaingdl]

few tools : doxygen, a tool in clang, cppclean by Google.

https://stackoverflow.com/questions/614794/detecting-superfluous-includes-in-c-c

[alaingdl]

I spend some time on this issue last week

• commenting #include "datatypes.hpp as @GillesDuvert suggested was not enough
• cppclean crashed after processing tens of files
• I did not succeed to run the tool in Clang

I plan to come back on OSX soon because my test-bed is back online :)

[alaingdl]

working today on that with the help of Eric, thanks.

 CC=gcc-8 CXX=g++-8 cmake .. -DPYTHON=OFF -DOPENMP=OFF  -DGEOTIFF=OFF  -DSHAPELIB=OFF -DREADLINEDIR=/usr/local/opt/readline  -DWXWIDGETS=OFF -DHDF=OFF -DHDF5=OFF -DUDUNITS2=OFF  -DPSLIB=OFF -DGRIB=off -DGLPK=OFF -DGRAPHICSMAGICK=off

gives :

[...]
duplicate symbol typeinfo for Data_<SpDLong64> in:
    CMakeFiles/gdl.dir/datatypes.cpp.o
    CMakeFiles/gdl.dir/ifmt.cpp.o
[...]
duplicate symbol typeinfo name for Data_<SpDLong64> in:
    CMakeFiles/gdl.dir/datatypes.cpp.o
    CMakeFiles/gdl.dir/ofmt.cpp.o
[...]

[alaingdl]

maybe it is related to lines around 217 in datatypes.hpp with :

#    if defined(__clang__)
#      pragma clang diagnostic ignored "-Wunsupported-friend"
[...]

[GillesDuvert]

The C++ code is indeed a bit convoluted. Splitting the huge datatypes file was too easy to be honest :smile: It seems that some linkers are less regarding than others, because indeed in the current state compiling two .cpp files create duplicate symbols for the same things. When the linker is accomodating, things go well and the fact that Data_<> is defined twice is transparent as the definition is the same. But this definitely needs to be sorted out.

[slayoo]

should we then wait with rc3 till solving it?

[alaingdl]

Ok for me. Wait rc4 ! I would say, as long as it's compiling fine for me on 10.14.4 with CLang on the same machine, and CLang is present by default, it is more or less non blocking.

The problem with 10.15.4 and X11 #780 is more annoying for me

[GillesDuvert]

Still no wxWidgets on MacOS?

[alaingdl]

I don't know ! I have no way to test. I don't have one physically. Just I use remotely some OSX, and just have some problems to have basic tcl-tk (zenity) or others tools. Even IDL (& widgets in IDL) remotely is bad on OSX for me (except basic plots) through the network.

[GillesDuvert]

how about removing template<class> class Data_; at line 279 in basegdl.hpp ?

[alaingdl]

I can try, but later ! this evening

[slayoo]

how about removing template<class> class Data_; at line 279 in basegdl.hpp ?

FWIW, compilation OK on my Debian laptop with gcc 8.3

[alaingdl]

(waiting from my oven)

not working for me :( (osx + gcc version 8.2.0 (Homebrew GCC 8.2.0) )

[GillesDuvert]

not working means what? does not compile (should, as it compiles with linux+gcc) or has the same duplicate symbol problem as above?

[alaingdl]

Sorry. Same duplicate symbols (compilation goes to the end)

[GillesDuvert]

everything is explained here. Now, will try some new tests...

[GillesDuvert]

@acoulais, coudl you try with:

1. instantiate_templates.hpp as such:

#ifndef INSTANTIATE_TEMPLATES_HPP_
#define INSTANTIATE_TEMPLATES_HPP_

inline template class Data_< SpDByte>;
inline template class Data_< SpDInt>;
inline template class Data_< SpDUInt>;
inline template class Data_< SpDLong>;
inline template class Data_< SpDULong>;
inline template class Data_< SpDLong64>;
inline template class Data_< SpDULong64>;
inline template class Data_< SpDPtr>;
inline template class Data_< SpDFloat>;
inline template class Data_< SpDDouble>;
inline template class Data_< SpDString>;
inline template class Data_< SpDObj>;
inline template class Data_< SpDComplex>;
inline template class Data_< SpDComplexDbl>;
#endif

and 2) the end of datatypes.cpp modified such as:

  return Real2Int<SizeT,double>(real((*this)[i]));
}
template class Data_< SpDByte>;
template class Data_< SpDInt>;
template class Data_< SpDUInt>;
template class Data_< SpDLong>;
template class Data_< SpDULong>;
template class Data_< SpDLong64>;
template class Data_< SpDULong64>;
template class Data_< SpDPtr>;
template class Data_< SpDFloat>;
template class Data_< SpDDouble>;
template class Data_< SpDString>;
template class Data_< SpDObj>;
template class Data_< SpDComplex>;
template class Data_< SpDComplexDbl>;

//#include "instantiate_templates.hpp"

pls notice the #include instantiate template is commented out :smile:

[alaingdl]

I think I made the tests as carefully as I can (on a new copy of the master)

• on the CLang side, I received 14 warnings !

/Users/coulais/GDL/gdl-1.0.0rc2-git200611CMake/src/instantiate_templates.hpp:33:1: warning: ignoring 'inline' keyword
      on explicit template instantiation [-Wstatic-inline-explicit-instantiation]
inline template class Data_< SpDComplex>;
^

but the compilation succeed.

• on the GCC side, no

duplicate symbol typeinfo name for Data_<SpDLong64> in:
    CMakeFiles/gdl.dir/datatypes.cpp.o
    CMakeFiles/gdl.dir/ofmt.cpp.o
duplicate symbol typeinfo for Data_<SpDLong64> in:
    CMakeFiles/gdl.dir/datatypes.cpp.o
    CMakeFiles/gdl.dir/ofmt.cpp.o
ld: 252 duplicate symbols for architecture x86_64
collect2: error: ld returned 1 exit status

[GillesDuvert]

Too bad. In retrospect, according to documentation above the code as it was should never have caused trouble to begin with, since the so-called 'Borland model' is imlemented by GCC on Mac OS X. so, what about removing

#if defined(__GNUC__) && !defined(__INTEL_COMPILER)
// by default intel C++ defines __GNUC__
#pragma interface
#endif

at lines 29-32 from datatypes.hpp

(i precise that every of the above changes produces always a correct compilation and code on my side, linux+gcc 8.4.0)

[alaingdl]

Yes, I played in this region yesterday, tryng to add similar flags than clang ones

I will try now

[alaingdl]

nop :-1: before, it was only 252 duplicate, now : ld: 1260 duplicate symbols for architecture x86_64

what is strange for me is that we have other machines with GCC 7, 8, 9 and no problem. Only problem here with this special compiler :( Since it is provided by Brew (cellar ..) may be it needs an update ? Or maybe a flag is missing, of CMake don't set it up well ?

[GillesDuvert]

Yes, I would say the GDL code passes almost everywhere because the compilers are quite friendly. The template instantiation duplication is between datatypes.cpp and ofmt.cpp etc. This can probably be cured by declaring 'extern' instead of 'inline' in instantiate_templates and remove from datatypes.hpp the definitions that are in conflict, while providing them for all the code that refer to them. Some work.