marian-dev icon indicating copy to clipboard operation
marian-dev copied to clipboard

Which protobuf version does USE_ONNX require/support?

Open alvations opened this issue 3 years ago • 3 comments

When trying to install marian with -DUSE_ONNX, it didn't compile.

Dependencies Installation

With a fresh ubuntu instance:

sudo apt update

sudo apt install git cmake build-essential libboost-system-dev libprotobuf17 protobuf-compiler libprotobuf-dev openssl libssl-dev libgoogle-perftools-dev

sudo apt install 

pip install onnxruntime

git clone https://github.com/marian-nmt/marian
mkdir marian/build
cd marian/build

Ubuntu version:

~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.4 LTS
Release:	20.04
Codename:	focal

Context

  • Marian version: v1.11.0

  • CMake command:

cmake .. -DUSE_STATIC_LIBS=on -DUSE_SENTENCEPIECE=on -DUSE_FBGEMM=on -DCOMPILE_CUDA=off -DCOMPILE_CPU=on -DUSE_ONNX=ON
  • Log file: Error seems to come from the protobuf not playing nice with onnx
 80%] Building CXX object src/CMakeFiles/marian.dir/3rd_party/ExceptionWithCallStack.cpp.o
[ 81%] Building CXX object src/CMakeFiles/marian.dir/3rd_party/onnx/protobuf/onnx-ml.pb-wrapper.cpp.o
In file included from /home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb.cc:4,
                 from /home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb-wrapper.cpp:29:
/home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb.h:12:2: error: #error This file was generated by a newer version of protoc which is
   12 | #error This file was generated by a newer version of protoc which is
      |  ^~~~~
/home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb.h:13:2: error: #error incompatible with your Protocol Buffer headers. Please update
   13 | #error incompatible with your Protocol Buffer headers. Please update
      |  ^~~~~
/home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb.h:14:2: error: #error your headers.
   14 | #error your headers.
      |  ^~~~~
In file included from /home/ubuntu/marian/src/3rd_party/sentencepiece/third_party/protobuf-lite/google/protobuf/io/coded_stream.h:135,
                 from /home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb.h:23,
                 from /home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb.cc:4,
                 from /home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb-wrapper.cpp:29:
/home/ubuntu/marian/src/3rd_party/sentencepiece/third_party/protobuf-lite/google/protobuf/stubs/common.h:111: error: "GOOGLE_PROTOBUF_MIN_LIBRARY_VERSION" redefined [-Werror]
  111 | #define GOOGLE_PROTOBUF_MIN_LIBRARY_VERSION 3006001
      | 
In file included from /home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb.h:10,
                 from /home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb.cc:4,
                 from /home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb-wrapper.cpp:29:
/usr/include/google/protobuf/port_def.inc:307: note: this is the location of the previous definition
  307 | #define GOOGLE_PROTOBUF_MIN_LIBRARY_VERSION 3011000
      | 
In file included from /usr/include/google/protobuf/descriptor.h:65,
                 from /usr/include/google/protobuf/generated_message_reflection.h:47,
                 from /home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb.h:30,
                 from /home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb.cc:4,
                 from /home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb-wrapper.cpp:29:
/usr/include/google/protobuf/port_def.inc:74:2: error: #error PROTOBUF_DEPRECATED was previously defined
   74 | #error PROTOBUF_DEPRECATED was previously defined
      |  ^~~~~
In file included from /usr/include/google/protobuf/metadata.h:42,
                 from /usr/include/google/protobuf/generated_message_reflection.h:49,
                 from /home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb.h:30,
                 from /home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb.cc:4,
                 from /home/ubuntu/marian/src/3rd_party/onnx/protobuf/onnx-ml.pb-wrapper.cpp:29:
/usr/include/google/protobuf/unknown_field_set.h:289:26: error: ‘google::protobuf::io::EpsCopyOutputStream’ has not been declared
  289 |       uint8* target, io::EpsCopyOutputStream* stream) const;
      |                        

alvations avatar Jun 09 '22 04:06 alvations

I don't know which versions are supported, but this may be potentially helpful: https://github.com/marian-nmt/marian-dev/issues/900#issuecomment-1015127934

snukky avatar Jun 11 '22 08:06 snukky