Dependency conflict with latest grpc/protobuf when using opentelemetry-exporter-otlp-proto-grpc

[vraychev]

Describe your environment

OS: does not matter Python version: 3.12 SDK version: 1.32.1 API version: 1.32.1

What happened?

The opentelemetry-exporter-otlp-proto-grpc==1.32.1 pip package imports opentelemetry-proto==1.33.0.dev, which forces protobuf < 6.0 here: https://github.com/open-telemetry/opentelemetry-python/blob/main/opentelemetry-proto/pyproject.toml#L30

Unfortunately, this means one cannot use recent versions of grpc (e.g. 1.72), because they use protobuf==6.30.2, which is incompatible.

Steps to Reproduce

Create requirements.txt with

... opentelemetry-api>=1.32.1 opentelemetry-instrumentation-grpc opentelemetry-sdk>=1.32.1 opentelemetry-exporter-otlp-proto-grpc>=1.32.1 protobuf==6.30.2

fails with errors like:

ERROR: Cannot install -r requirements.txt (line 16), -r requirements.txt (line 17), -r requirements.txt (line 28), opentelemetry-exporter-otlp-proto-grpc and protobuf==6.30.2 because these package versions have conflicting dependencies.

Expected Result

I would expect the constraint to be relaxed to say < 7.0

https://github.com/open-telemetry/opentelemetry-python/blob/main/opentelemetry-proto/pyproject.toml#L30

Actual Result

Install fails.

Additional context

No response

Would you like to implement a fix?

None

[vraychev]

It is supposed to work without regenerating code. I know for sure generating code https://protobuf.dev/support/cross-version-runtime-guarantee/#major

However, when updating other components, I noticed the runtime doesn't accept older gencode.

[vraychev]

I tried to fork opentelemetry-proto and relaxed the requirement in https://github.com/open-telemetry/opentelemetry-python/blob/main/opentelemetry-proto/pyproject.toml#L30

to "protobuf>=5.0, < 7.0", and in line with published the proto spec, the relaxed version requirements work in my setup: https://protobuf.dev/support/cross-version-runtime-guarantee/#major

So, I think the fix should be simple and at least in my tests, works well.

[emdneto]

I tried to fork opentelemetry-proto and relaxed the requirement in https://github.com/open-telemetry/opentelemetry-python/blob/main/opentelemetry-proto/pyproject.toml#L30

to "protobuf>=5.0, < 7.0", and in line with published the proto spec, the relaxed version requirements work in my setup: https://protobuf.dev/support/cross-version-runtime-guarantee/#major

So, I think the fix should be simple and at least in my tests, works well.

Although tests pass, there's no guarantee I guess. What I understand is that the compatibility window starts from 6.30 release

[dws]

In order to mimic #3958 I'd suggest changing the title of this issue to "Add support for Protobuf 6".

[vraychev]

We are using forked version in production, so your mileage may vary, but from the perspective of claimed compatibility and some evidence, the dependency can be relaxed. I think this is in contrary to previous much more painful Protobuf 5 update.

[lukasbindreiter]

We are also using both grpc and opentelemetry-python and would therefore appreciate if Protobuf v6 could be officially supported soon.

I haven't tested it with the otel messages yet, but in our own proto message files the migration from v5 to v6 was without any issues, no changes were needed - so just relaxing the dependency constraint here could already be enough.

[wardvandriessche-centrica]

Running into issues too with packages that only support protobuf ~= 6.30. I cannot use opentelemetry anymore :-(

[maroux]

opened #4639 with better title.

[yalishanda42]

Just to ask, when to expect a new version to be released in PyPI? As a temporarily workaround I am currently pinning the main branch of all six packages needed from the repo, but it would be good to have the protobuf dependency issue published soon 😄