Could NOT find Protobuf (missing: Protobuf_PROTOC_EXECUTABLE)

[Raviteja1996]

Hi, when I am doing cmake in the build directory of lbann, initially I got protobuf-config.cmake not found error. So I gave a path of the protobuf-config.cmake file to Protobuf_DIR but then it started to show another error. I am attaching the error log for cmake command.

lbann_errors.txt

[benson31]

You have two options: Either you can rebuild protobuf in Debug or Release mode (it's probably in RelWithDebInfo mode) or you can pass -DLBANN_USE_PROTOBUF_MODULE=ON to LBANN's CMake.

[Raviteja1996]

Hi I tried the second way, using -DLBANN_USE_PROTOBUF_MODULE=ON and it worked fine . But when I do a "make" in the build directory its showing a lot of errors related to protobuf. I am attaching the error log file.

error.txt

[Raviteja1996]

Hi, any update on the error from the above post.

[timmoon10]

I have a hypothesis. When we build lbann.pb.h, we use the CMake variable protobuf::protoc

https://github.com/LLNL/lbann/blob/c2e7f2b624ca7a7cddc8b6482028b5e289893e9c/src/proto/CMakeLists.txt#L13

However, this is not set by the FindProtobuf module. CMake finds an old version of protoc that lives somewhere in @Raviteja1996's path, which is incompatible with the Protobuf header files in Protobuf_DIR. Ideas @benson31?

[benson31]

It is set by the module as of CMake 3.10 -- I should tick up the minimum version. Anyway, try a newer CMake. Also, try a clean build directory.

[benson31]

Nevermind, I see you're using CMake 3.13 -- good! I'm wondering if you had some old cached state from a previous run before you tried -DLBANN_USE_PROTOBUF_MODULE=ON. If that's the case, remove CMakeCache.txt (and CMakeFiles/ for good measure) and try again.

[Raviteja1996]

I do clean the entire build directory every time before I do a 'cmake'. But still I am still facing the problem.

[benson31]

Hmmm. This might be ugly, but it would be good if we could get some more information out of the build. Can you please try the following:

1. Add -DCMAKE_CXX_FLAGS=-v to your CMake invocation. (I want to know what CMake thinks your protobuf dir should be and what the compiler is finding when it resolves include paths; this flag will include that (assuming you're using some GCC/Clang-y thing).)
2. Save the output of CMake and post it here.
3. Run make VERBOSE=1 and post the output of that here as well. I'm hoping this will print the path to the protoc compiler as well, but I'm not sure.

The other thing to check would be to just see if you can compile the lbann.pb.o file yourself after configuring LBANN. The compile line should be something like (assuming gcc):

g++ -std=c++11 /path/to/protobuf/include -c lbann.pb.cc

If that fails, it probably indicates that the right protoc is not being picked up by CMake.

[Raviteja1996]

I am posting 2 files. First one was the output of CMake and second was error output of make VERBOSE=1. ActuaLly I was getting again the same error as I mentioned above with error.txt.

CMAKE : CMake.txt

MAKE : make_err.txt

[benson31]

This is strange. The salient line from the CMake output is:

  Protobuf compiler version 3.0.0 doesn't match library version 3.7.0

From make_err.txt, it seems that it's finding /usr/bin/protoc for the compiler and /usr/local/lib/libprotobuf.so for the library. Do you have two separate installs, one in /usr (perhaps from a system package manager, yum, apt, whatever?) and the other in /usr/local? That is, does /usr/local/bin/protoc exist? If so, you might try specifying -DPROTOBUF_DIR=/usr/local in the CMake invocation.

[Raviteja1996]

Hi, I tried the way you mentioned above. But there is still seems to be error while doing make. Before, without starting only the error used to come but now I am getting the error on 5%. I am attaching the logs for both cmake and make.

CMAKE : cmake ../ -DPROTOBUF_DIR=/usr/local -DLBANN_USE_PROTOBUF_MODULE=ON -DCMAKE_CXX_FLAGS=-v CMake.txt

MAKE : make VERBOSE=1 make_error.txt

[Raviteja1996]

Hi any update/way to solve about the above error.

[benson31]

I'm not sure what the underlying problem is, but it definitely seems protobuf-centric. My suggestion would be to try a simple superbuild to build Protobuf and LBANN. Basically,

cmake \
    -DLBANN_SB_BUILD_PROTOBUF=ON \
    \
    -DLBANN_SB_BUILD_LBANN=ON \
    <Any other options you want to pass to LBANN> \
    ${LBANN_HOME}/superbuild

make -j

This will pull down and build the latest protobuf and should link directly to that version. I built this way this morning and the build went through just fine.

[den-run-ai]

same problem here: https://github.com/LLNL/lbann/issues/1442