CNTK icon indicating copy to clipboard operation
CNTK copied to clipboard
verified publisher icon microsoft

Build error undefined reference to `google::protobuf

Open DavidBrayford opened this issue 7 years ago • 3 comments

After moving to version 2.5.1 I am now getting following build errors related to protobuf. I've built protobuf-3.1.0 and successfully built version 2.4 with the same version of protobuf.

building /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/bin/cppevalclient for Linux with build type release /lrz/sys/intel/studio2017_u4/impi/2017.3.196/bin/mpicxx -rdynamic -L/home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/lib -L/lrz/sys/intel/studio2017_u4/compilers_and_libraries_2017.4.196/linux/mkl//lib -L/usr/lib -L/home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/opencv-3.1.0/build/release_icc/lib -L/home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/opencv-3.1.0/build/release_icc/release/lib -L/home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/opencv-3.1.0/build/release_icc/lib -L/home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/opencv-3.1.0/build/release_icc/release/lib -Wl,-rpath,'$ORIGIN/../lib' -Wl,-rpath,/lrz/sys/intel/studio2017_u4/compilers_and_libraries_2017.4.196/linux/mkl//lib -Wl,-rpath,/usr/lib -Wl,-rpath,/home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/opencv-3.1.0/build/release_icc/lib -Wl,-rpath,/home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/opencv-3.1.0/build/release_icc/release/lib -Wl,-rpath,/home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/opencv-3.1.0/build/release_icc/lib -Wl,-rpath,/home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/opencv-3.1.0/build/release_icc/release/lib -o /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/bin/cppevalclient /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/.build/Source/../Examples/Evaluation/LegacyEvalDll/CPPEvalClient/CPPEvalClient.o -lm -liomp5 -lpthread -lmklml_intel -lCntk.Eval-2.5.1 -lCntk.Math-2.5.1 -lCntk.PerformanceProfiler-2.5.1 -lmultiverso /usr/bin/ld: skipping incompatible /usr/lib/libpthread.so when searching for -lpthread /usr/bin/ld: skipping incompatible /usr/lib/libdl.so when searching for -ldl /usr/bin/ld: skipping incompatible /usr/lib/librt.so when searching for -lrt /usr/bin/ld: skipping incompatible /usr/lib/libpthread.so when searching for -lpthread /usr/bin/ld: skipping incompatible /usr/lib/libm.so when searching for -lm /usr/bin/ld: skipping incompatible /usr/lib/libc.so when searching for -lc /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/lib/libCntk.Eval-2.5.1.so: undefined reference to google::protobuf::internal::WireFormatLite::WriteStringMaybeAliased(int, std::string const&, google::protobuf::io::CodedOutputStream*)' /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/lib/libCntk.Eval-2.5.1.so: undefined reference to google::protobuf::io::CodedOutputStream::WriteStringWithSizeToArray(std::string const&, unsigned char*)' /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/lib/libCntk.Eval-2.5.1.so: undefined reference to google::protobuf::Message::GetTypeName() const' /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/lib/libCntk.Eval-2.5.1.so: undefined reference to google::protobuf::MessageFactory::InternalRegisterGeneratedFile(char const*, void ()(std::string const&))' /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/lib/libCntk.Eval-2.5.1.so: undefined reference to google::protobuf::MessageLite::AppendToString(std::string*) const' /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/lib/libCntk.Eval-2.5.1.so: undefined reference to google::protobuf::internal::WireFormatLite::WriteBytes(int, std::string const&, google::protobuf::io::CodedOutputStream)' /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/lib/libCntk.Eval-2.5.1.so: undefined reference to google::protobuf::internal::WireFormatLite::WriteBytesMaybeAliased(int, std::string const&, google::protobuf::io::CodedOutputStream*)' /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/lib/libCntk.Eval-2.5.1.so: undefined reference to google::protobuf::MessageLite::InitializationErrorString() const' /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/lib/libCntk.Eval-2.5.1.so: undefined reference to google::protobuf::Message::ShortDebugString() const' /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/lib/libCntk.Eval-2.5.1.so: undefined reference to google::protobuf::internal::fixed_address_empty_string' /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/lib/libCntk.Eval-2.5.1.so: undefined reference to google::protobuf::internal::WireFormatLite::WriteString(int, std::string const&, google::protobuf::io::CodedOutputStream*)' /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/lib/libCntk.Eval-2.5.1.so: undefined reference to google::protobuf::DescriptorPool::FindFileByName(std::string const&) const' /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/lib/libCntk.Eval-2.5.1.so: undefined reference to google::protobuf::internal::ArenaStringPtr::AssignWithDefault(std::string const*, google::protobuf::internal::ArenaStringPtr)' /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/lib/libCntk.Eval-2.5.1.so: undefined reference to google::protobuf::internal::WireFormatLite::ReadBytes(google::protobuf::io::CodedInputStream*, std::string*)' /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/lib/libCntk.Eval-2.5.1.so: undefined reference to `google::protobuf::Message::InitializationErrorString() const' collect2: error: ld returned 1 exit status

DavidBrayford avatar Aug 02 '18 10:08 DavidBrayford

I am also getting the following boost errors:

/lrz/sys/intel/studio2017_u4/impi/2017.3.196/bin/mpicxx -c Source/Readers/HTKDeserializers/HTKDeserializer.cpp -o /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/.build/Source/Readers/HTKDeserializers/HTKDeserializer.o -DONNX_NAMESPACE=onnx -DONNX_ML=1 -DHAS_MPI=1 -D_POSIX_SOURCE -D_XOPEN_SOURCE=600 -D__USE_XOPEN2K -std=c++14 -DCUDA_NO_HALF -D__CUDA_NO_HALF_OPERATORS__ -DCPUONLY -DUSE_MKL -DNDEBUG -DNO_SYNC -DCNTK_PARALLEL_TRAINING_SUPPORT -DASGD_PARALLEL_SUPPORT -mavx512f -mavx512cd -mavx512er -mavx512pf -D_GLIBCXX_USE_CXX11_ABI=0 -DKALDI_DOUBLEPRECISION=0 -DHAVE_POSIX_MEMALIGN -DHAVE_EXECINFO_H=1 -DHAVE_CXXABI_H -DHAVE_ATLAS -DHAVE_OPENFST_GE_10400 -DCNTK_VERSION="2.5.1" -DCNTK_VERSION_BANNER="2.5.1+" -DCNTK_COMPONENT_VERSION="2.5.1" -DUSE_ZIP -msse4.1 -mssse3 -fopenmp -fpermissive -fPIC -Werror -fcheck-new -Wno-deprecated-declarations -Wno-error=literal-suffix -g -O4 -ISource/Common/Include -ISource/CNTKv2LibraryDll -ISource/CNTKv2LibraryDll/API -ISource/CNTKv2LibraryDll/Generated/Linux -ISource/CNTKv2LibraryDll/proto -ISource/../Examples/Extensibility/CPP -ISource/Math -ISource/CNTK -ISource/ActionsLib -ISource/ComputationNetworkLib -ISource/SGDLib -ISource/SequenceTrainingLib -ISource/CNTK/BrainScript -ISource/Readers/ReaderLib -ISource/PerformanceProfilerDll -I/home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/protobuf-3.6.1/build/gcc/include -ISource/../external/gsl/include -ISource/CNTKv2LibraryDll/proto/onnx -ISource/CNTKv2LibraryDll/proto/onnx/onnx_repo -I/lrz/sys/intel/studio2017_u4/compilers_and_libraries_2017.4.196/linux/mkl//include -I/home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/kaldi/src -I/home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/kaldi/tools/ATLAS/include -I/home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/kaldi/tools/openfst/include -I/home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/halide/Halide/build/include -I/lrz/sys/libraries/boost/1.58_gcc/include -I/usr/include -I/usr/lib/libzip/include -I/home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/opencv-3.1.0/build/release_icc/include -I/home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/opencv-3.1.0/build/release_icc/include -ISource/1BitSGD -ISource/Multiverso/include -ITests/EndToEndTests/CNTKv2Library/Common -I/lrz/sys/libraries/boost/1.58_gcc/include -ISource/Readers/CNTKTextFormatReader -MD -MP -MF /home/hpc/pr28fa/di72giz/CNTK_COOLMUC3/CNTK_2.5.1/CNTK/build/.build/Source/Readers/HTKDeserializers/HTKDeserializer.d Source/../Tests/UnitTests/ReaderTests/ReaderLibTests.cpp: In lambda function: Source/../Tests/UnitTests/ReaderTests/ReaderLibTests.cpp:1277:17: error: ‘BOOST_TEST’ was not declared in this scope BOOST_TEST((s.m_data.empty() && numberOfMinibatches == 1 && numberOfSamples == 0)); ^~~~~~~~~~ Source/../Tests/UnitTests/ReaderTests/ReaderLibTests.cpp:1277:17: note: suggested alternative: ‘BOOST_TEST_L’ BOOST_TEST((s.m_data.empty() && numberOfMinibatches == 1 && numberOfSamples == 0)); ^~~~~~~~~~ BOOST_TEST_L Source/../Tests/UnitTests/ReaderTests/ReaderLibTests.cpp:1289:17: error: ‘BOOST_TEST’ was not declared in this scope BOOST_TEST(((maxMBSize / p.sequenceLength) == numSequencesInTheLastMB)); ^~~~~~~~~~ Source/../Tests/UnitTests/ReaderTests/ReaderLibTests.cpp:1289:17: note: suggested alternative: ‘BOOST_TEST_L’ BOOST_TEST(((maxMBSize / p.sequenceLength) == numSequencesInTheLastMB)); ^~~~~~~~~~ BOOST_TEST_L Source/../Tests/UnitTests/ReaderTests/ReaderLibTests.cpp:1293:17: error: ‘BOOST_TEST’ was not declared in this scope BOOST_TEST((ceil(lastMBSize1.0 / p.sequenceLength) == numSequencesInTheLastMB)); ^~~~~~~~~~ Source/../Tests/UnitTests/ReaderTests/ReaderLibTests.cpp:1293:17: note: suggested alternative: ‘BOOST_TEST_L’ BOOST_TEST((ceil(lastMBSize1.0 / p.sequenceLength) == numSequencesInTheLastMB)); ^~~~~~~~~~ BOOST_TEST_L Source/../Tests/UnitTests/ReaderTests/ReaderLibTests.cpp:1296:13: error: ‘BOOST_TEST’ was not declared in this scope BOOST_TEST((numberOfSamples <= epochSize || (numberOfSamples - epochSize) < p.sequenceLength)); ^~~~~~~~~~ Source/../Tests/UnitTests/ReaderTests/ReaderLibTests.cpp:1296:13: note: suggested alternative: ‘BOOST_TEST_L’ BOOST_TEST((numberOfSamples <= epochSize || (numberOfSamples - epochSize) < p.sequenceLength)); ^~~~~~~~~~ BOOST_TEST_L

DavidBrayford avatar Aug 06 '18 05:08 DavidBrayford

After updating my local repository the I've been succesfully able to build CNTK. However, if I don't remove -werror flag from the Makefile I get the errors below.

Source/Readers/Kaldi2Reader/KaldiSequenceTrainingDerivative.cpp:228:24: error: ISO C++ forbids comparison between pointer and integer [-Werror=permissive] if (m_aliReader == false || m_denlatReader == false) ^~~~~ Source/Readers/Kaldi2Reader/KaldiSequenceTrainingDerivative.cpp:228:51: error: ISO C++ forbids comparison between pointer and integer [-Werror=permissive] if (m_aliReader == false || m_denlatReader == false) ^~~~~ Source/Readers/Kaldi2Reader/KaldiSequenceTrainingDerivative.cpp: In instantiation of ‘bool Microsoft::MSR::CNTK::KaldiSequenceTrainingDerivative<ElemType>::HasResourceForDerivative(const wstring&) const [with ElemType = float; std::__cxx11::wstring = std::__cxx11::basic_string<wchar_t>]’: Source/Readers/Kaldi2Reader/KaldiSequenceTrainingDerivative.cpp:243:16: required from here Source/Readers/Kaldi2Reader/KaldiSequenceTrainingDerivative.cpp:228:21: error: ISO C++ forbids comparison between pointer and integer [-Werror=permissive] if (m_aliReader == false || m_denlatReader == false) ~~~~~~~~~~~~^~~~~~~~ Source/Readers/Kaldi2Reader/KaldiSequenceTrainingDerivative.cpp:228:48: error: ISO C++ forbids comparison between pointer and integer [-Werror=permissive] if (m_aliReader == false || m_denlatReader == false)

DavidBrayford avatar Aug 09 '18 05:08 DavidBrayford

Hello, I'm facing a similar problem with "undefined reference to `google::protobuf". I'm compiling CNTK2.7 from the source. I used protobuf-3.1.0 and cmake3.16.2. When I "make -j all", it returns:

"data/zxy/DL_framework/cntk/build/release/lib/libCntk.Eval-2.6.so: undefined reference to google::protobuf::internal::WireFormatLite::ReadBytes(google::protobuf::io::CodedInputStream*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)' /data/zxy/DL_framework/cntk/build/release/lib/libCntk.Eval-2.6.so: undefined reference to google::protobuf::Message::GetTypeNameabi:cxx11 const' /data/zxy/DL_framework/cntk/build/release/lib/libCntk.Eval-2.6.so: undefined reference to google::protobuf::io::CodedOutputStream::WriteStringWithSizeToArray(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned char*)' /data/zxy/DL_framework/cntk/build/release/lib/libCntk.Eval-2.6.so: undefined reference to google::protobuf::internal::fixed_address_empty_string[abi:cxx11]' /data/zxy/DL_framework/cntk/build/release/lib/libCntk.Eval-2.6.so: undefined reference to google::protobuf::MessageLite::AppendToString(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) const' /data/zxy/DL_framework/cntk/build/release/lib/libCntk.Eval-2.6.so: undefined reference to google::protobuf::Message::InitializationErrorStringabi:cxx11 const' /data/zxy/DL_framework/cntk/build/release/lib/libCntk.Eval-2.6.so: undefined reference to google::protobuf::internal::NameOfEnum[abi:cxx11](google::protobuf::EnumDescriptor const*, int)' /data/zxy/DL_framework/cntk/build/release/lib/libCntk.Eval-2.6.so: undefined reference to google::protobuf::MessageLite::InitializationErrorStringabi:cxx11 const' /data/zxy/DL_framework/cntk/build/release/lib/libCntk.Eval-2.6.so: undefined reference to `google::protobuf::internal::WireFormatLite::WriteBytesMaybeAliased(int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, google::protobuf::io::CodedOutputStream*)' "

I wonder how to compile it correctly and how to solve this kind of problem.

shiningrain avatar Jan 07 '20 11:01 shiningrain