sklearn-onnx

sklearn-onnx copied to clipboard

Version: 1.10.0 Error:

Traceback (most recent call last):
  File "/home/circleci/repo/tests/modeling/export/test_export_model.py", line 5, in <module>
    from skl2onnx.common.data_types import FloatTensorType
  File "/home/circleci/.venv/lib/python3.9/site-packages/skl2onnx/__init__.py", line 15, in <module>
    from .convert import convert_sklearn, to_onnx, wrap_as_onnx_mixin  # noqa
  File "/home/circleci/.venv/lib/python3.9/site-packages/skl2onnx/convert.py", line 5, in <module>
    from .proto import get_latest_tested_opset_version
  File "/home/circleci/.venv/lib/python3.9/site-packages/skl2onnx/proto/__init__.py", line 8, in <module>
    from onnx import onnx_pb as onnx_proto  # noqa
  File "/home/circleci/.venv/lib/python3.9/site-packages/onnx/__init__.py", line 11, in <module>
    from onnx.external_data_helper import load_external_data_for_model, write_external_data_tensors, convert_model_to_external_data
  File "/home/circleci/.venv/lib/python3.9/site-packages/onnx/external_data_helper.py", line 14, in <module>
    from .onnx_pb import TensorProto, ModelProto
  File "/home/circleci/.venv/lib/python3.9/site-packages/onnx/onnx_pb.py", line 8, in <module>
    from .onnx_ml_pb2 import *  # noqa
  File "/home/circleci/.venv/lib/python3.9/site-packages/onnx/onnx_ml_pb2.py", line 33, in <module>
    _descriptor.EnumValueDescriptor(
  File "/home/circleci/.venv/lib/python3.9/site-packages/google/protobuf/descriptor.py", line 755, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

------------------------------------------------------------------------------
Ran 0 tests in 2.25s

Exited with code exit status 2

onnx does not support the latest version of protobuf yet. Until then, the only way is to use protobuf<4.0.

onnx does not support the latest version of protobuf yet. Until then, the only way is to use protobuf<4.0.

This makes sense, but how? Why does sklearn-onnx not declare a compatible version of protobuf as a dependency? We have skl2onnx pinned to version 1.11.1, but still encountered this issue. Specifically including protobuf="3.19" in our pipfile doesn't seem to resolve the issue. skl2onnx seems to be relying on google.protobuf, I'm not clear if this is an alias for protobuf or a different package.

Sklearn-onnx does not have dependency on protobuf but onnx does. You can see pr #868. It fixez CI by downgrading protobuf ans it seems to work.

sklearn-onnx does have a dependency defined for protobuf: https://github.com/onnx/sklearn-onnx/blob/main/requirements.txt

requirements.txt is read by setup.py and it's a dependency with the current package on PyPI.

I've filed issues with onnxruntime, onnx and see this here. It's surprising that there is no sense of urgency to fix this issue, which is a breaking change for many people. It seems like the earliest a fix will be available is weeks or months away.

The workaround of installing protobuf < 4 works but many don't know protobuf is being used by onnx.

Any news on this?

Having downgraded to protobuf = "3.20.1", I now get the following error message: TypeError: Couldn't build proto file into descriptor pool: duplicate file name (onnx/onnx-ml.proto)

EDIT: I was using Windows. Worked fine after moving my stuff into a Linux-based dev-container

I'm wondering if you installed onnx with protobuf 4 and then remove protobuf 4 to install protobuf 3. I'm wondering if some intermediate files were generated using the protobuf 4 and were not refreshed after downgraded protobuf. Is it possible to uninstall and reinstall onnx to make sure?

This issue is more related to onnx, see https://github.com/onnx/onnx/pull/4242.