tensorflow
pip_package .whl file not being created.
I am following the instructions in pip_package/README.md to get a whl file to install lingvo using pip3 with one change in the 'build.sh' file where I changed 'TF_NEED_CUDA' value from 0 to 1 for enabling GPU support. But when I run the command bash pip_package/build.sh inside the docker container the directory 'lingvo_pip_pkg_build' and whl files are not created. The build command also runs into a partial fail even though I get the output "INFO: Build completed, 1 test FAILED, 126 total actions" result finally. 1 out of 64 tests 'hyperparams_test' fails. the following is the log file from which the failure begins:
[ RUN ] ParamsTest.testToFromProto [ FAILED ] ParamsTest.testToFromProto [ RUN ] ParamsTest.testToStr [ OK ] ParamsTest.testToStr [ RUN ] ParamsTest.testToString [ OK ] ParamsTest.testToString [ RUN ] ParamsTest.testToText [ OK ] ParamsTest.testToText [ RUN ] ParamsTest.testTypeOverride [ OK ] ParamsTest.testTypeOverride [ RUN ] ParamsTest.test_session [ SKIPPED ] ParamsTest.test_session
ERROR: testToFromProto (main.ParamsTest) testToFromProto (main.ParamsTest)
Traceback (most recent call last): File "/root/.cache/bazel/_bazel_root/17eb95f0bc03547f4f1319e61997e114/sandbox/processwrapper-sandbox/966/execroot/main/bazel-out/k8-opt/bin/lingvo/core/hyperparams_test.runfiles/main/lingvo/core/hyperparams_test.py", line 342, in testToFromProto rebuilt_outer = _params.InstantiableParams.FromProto(outer.ToProto()) File "/root/.cache/bazel/_bazel_root/17eb95f0bc03547f4f1319e61997e114/sandbox/processwrapper-sandbox/966/execroot/main/bazel-out/k8-opt/bin/lingvo/core/hyperparams_test.runfiles/main/lingvo/core/hyperparams.py", line 442, in ToProto return _ToParam(self) File "/root/.cache/bazel/_bazel_root/17eb95f0bc03547f4f1319e61997e114/sandbox/processwrapper-sandbox/966/execroot/main/bazel-out/k8-opt/bin/lingvo/core/hyperparams_test.runfiles/main/lingvo/core/hyperparams.py", line 439, in _ToParam param_pb.items[k].CopyFrom(_ToParamValue(v)) File "/root/.cache/bazel/_bazel_root/17eb95f0bc03547f4f1319e61997e114/sandbox/processwrapper-sandbox/966/execroot/main/bazel-out/k8-opt/bin/lingvo/core/hyperparams_test.runfiles/main/lingvo/core/hyperparams.py", line 402, in _ToParamValue param_pb.param_val.CopyFrom(_ToParam(val)) File "/root/.cache/bazel/_bazel_root/17eb95f0bc03547f4f1319e61997e114/sandbox/processwrapper-sandbox/966/execroot/main/bazel-out/k8-opt/bin/lingvo/core/hyperparams_test.runfiles/main/lingvo/core/hyperparams.py", line 439, in _ToParam param_pb.items[k].CopyFrom(_ToParamValue(v)) File "/root/.cache/bazel/_bazel_root/17eb95f0bc03547f4f1319e61997e114/sandbox/processwrapper-sandbox/966/execroot/main/bazel-out/k8-opt/bin/lingvo/core/hyperparams_test.runfiles/main/lingvo/core/hyperparams.py", line 406, in _ToParamValue param_pb.list_val.items.append(_ToParamValue(v)) AttributeError: 'google.protobuf.pyext._message.RepeatedCompositeCo' object has no attribute 'append'
Ran 19 tests in 0.282s
FAILED (errors=1, skipped=1)
Is the whl file not being built due to this error? How do I resolve this issue.
This is due to the recent code expecting a newer version of the protobuf library which has the ".append()" function for repeated fields.
Which version of protobuf is installed? setup.py has "protobuf>=3.7,<4", but we may need to update it to something newer now.
Try pip3 install protobuf==3.8 and then re-running everything?
Edit: Oh hm, protobuf 3.11 is certainly higher than 3.8. Not sure what's going on.
I'd try following the instructions here line by line:
lingvo/pip_package/README.md, if you aren't already (try to use the Docker container provided) - I've only verified that building the pip package works using those instructions.