How to have the correction version of protoc when building with nightly or 2.3.0rc0 ?

[jrabary]

I don't know if this issue is related to version of protocol buffer used in the current existing image tensorflow/tensorflow:2.2.0-custom-op-gpu-ubuntu16 to build custom ops.

I'm trying to build GRPC ops from seed_rl https://github.com/google-research/seed_rl/tree/master/grpc with tf-2.3.0rc0 or tf-nightly. I reported the issue here https://github.com/google-research/seed_rl/issues/14 as well.

I tested two options: upgrade the version of TensorFlow of tensorflow/tensorflow:2.2.0-custom-op-gpu-ubuntu16 from 2.2 to 2.3.0rc0 or use tensorflow/tensorflow:nightly-custom-op-gpu-ubuntu16.

My starting point was the docker file provided to builds the ops for tf-2.2

FROM tensorflow/tensorflow:2.2.0-custom-op-gpu-ubuntu16 as grpc_compile

RUN git clone https://github.com/tensorflow/custom-op.git
WORKDIR custom-op

RUN ./configure.sh

RUN echo '\n\
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")\n\
\n\
http_archive(\n\
    name = "com_github_grpc_grpc",\n\
    urls = [\n\
        "https://github.com/grpc/grpc/archive/ac1c5de1b36da4a1e3d72ca40b0e43f24266121a.tar.gz",\n\
    ],\n\
    strip_prefix = "grpc-ac1c5de1b36da4a1e3d72ca40b0e43f24266121a",\n\
)\n\
\n\
load("@com_github_grpc_grpc//bazel:grpc_deps.bzl", "grpc_deps")\n\
grpc_deps()\n\
load("@com_github_grpc_grpc//bazel:grpc_extra_deps.bzl", "grpc_extra_deps")\n\
grpc_extra_deps()' >> WORKSPACE

ADD grpc/ grpc/
RUN bazel build grpc:ops/grpc.so grpc:service_py_proto --incompatible_remove_legacy_whole_archive=0

ADD . /seed_rl
RUN cp bazel-bin/grpc/ops/grpc.so /seed_rl/grpc/grpc_cc.so
RUN cp bazel-bin/grpc/service_pb2.py /seed_rl/grpc/service_pb2.py
WORKDIR /seed_rl/
RUN pip3 install tensorflow-gpu==2.2.0
RUN PYTHONPATH=/ python3 grpc/python/ops_test.py

Both of my attempts fail with

ERROR: /custom-op/grpc/BUILD.bazel:4:1: C++ compilation of rule '//grpc:ops/grpc.so' failed (Exit 1)
In file included from bazel-out/k8-opt/bin/grpc/service.grpc.pb.h:21:0,
                 from grpc/ops/grpc.cc:23:
bazel-out/k8-opt/bin/grpc/service.pb.h:17:2: error: #error This file was generated by an older version of protoc which is
 #error This file was generated by an older version of protoc which is
  ^~~~~
bazel-out/k8-opt/bin/grpc/service.pb.h:18:2: error: #error incompatible with your Protocol Buffer headers. Please
 #error incompatible with your Protocol Buffer headers. Please
  ^~~~~
bazel-out/k8-opt/bin/grpc/service.pb.h:19:2: error: #error regenerate this file with a newer version of protoc.
 #error regenerate this file with a newer version of protoc.
  ^~~~~

My understanding is that there is an incompatibility between the protoc version used to generate the tensorflow library and the protoc being used during this build. Is that the problem here and how can we fix this ?

[prouast]

Having the same issue.

[lespeholt]

What appears to work is using a custom protoc version in a build file, like: https://github.com/google-research/seed_rl/blob/master/docker/Dockerfile.grpc#L23

Should be fixed in this repo though.