Fails to build with Undefined Behavior Sanitizer (ubsan)
Add CXXFLAGS="-fsanitize=undefined" before cmake to enable ubsan (with relatively recent Clang or GCC). Build then fails with:
CMakeFiles/dhewm3.dir/tools/compilers/dmap/map.cpp.o:(.data+0xd258): undefined reference to `typeinfo for idGameEdit'
CMakeFiles/dhewm3.dir/tools/compilers/aas/AASBuild.cpp.o:(.data+0xa1b8): undefined reference to `typeinfo for idGameEdit'
CMakeFiles/dhewm3.dir/tools/compilers/aas/AASBuild.cpp.o:(.data+0xa2a8): undefined reference to `typeinfo for idGameEdit'
CMakeFiles/dhewm3.dir/tools/compilers/aas/AASBuild.cpp.o:(.data+0xab48): undefined reference to `typeinfo for idGameEdit'
CMakeFiles/dhewm3.dir/tools/compilers/aas/AASBuild.cpp.o:(.data+0xaea8): undefined reference to `typeinfo for idGameEdit'
CMakeFiles/dhewm3.dir/tools/compilers/aas/AASBuild.cpp.o:(.data+0xb7f8): more undefined references to `typeinfo for idGameEdit' follow
UBSan needs typeinfo and therefore vtable for all classes which are called. idGameEdit is declared and used in game executable but only implemented in game .so so it's vtable is incomplete. There's a FIXME comment about it in Game.h line 236.
Changing idGameEdit so it's pure abstract class in game and fully implemented in .so should work. However it's part of gameExport_t and therefore changing it would change game ABI, am I correct?
Is this a safe change?
the game ABI shouldn't be changed if not absolutely necessary, so I guess I can't do anything about this :-/
@turol: https://github.com/dhewm/dhewm3/tree/ubsan has a new cmake option to statically link the game code, so ubsan can be used. I'll probably merge that change to master eventually
Nice. Now someone (else) just needs to fix all of the problems. Preferably before the compiler writers decide that this particular UB makes for a nice optimization opportunity.