opentelemetry-python-contrib icon indicating copy to clipboard operation
opentelemetry-python-contrib copied to clipboard

"Failed to detach context" exception for grpc instrumentation

Open shiranbi opened this issue 1 year ago • 1 comments
trafficstars

open telemetry version - 0.44b0 python - 3.10 docker versions - ubuntu 20.04 running in Kubernetes cluster, on prem servers

The issue isn't reproduced systematically
many times i get the following exception (everything is fine except for the error printed)

Failed to detach context
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/opentelemetry/trace/__init__.py", line 573, in use_span
    yield span
  File "/usr/local/lib/python3.10/dist-packages/opentelemetry/sdk/trace/__init__.py", line 1046, in start_as_current_span
    yield span
  File "/usr/local/lib/python3.10/dist-packages/opentelemetry/instrumentation/grpc/_client.py", line 211, in _intercept_server_stream
    yield from invoker(request_or_iterator, metadata)
GeneratorExit

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/opentelemetry/context/__init__.py", line 163, in detach
    _RUNTIME_CONTEXT.detach(token)  # type: ignore
  File "/usr/local/lib/python3.10/dist-packages/opentelemetry/context/contextvars_context.py", line 50, in detach
    self._current_context.reset(token)  # type: ignore
ValueError: <Token var=<ContextVar name='current_context' default={} at 0x7fbf8d8b2bb0> at 0x7fbf4c4b9d40> was created in a different Context

there are related bugs but not regarding grpc instrumentation https://github.com/open-telemetry/opentelemetry-python-contrib/issues/904 https://github.com/open-telemetry/opentelemetry-python/issues/2606

shiranbi avatar Mar 20 '24 05:03 shiranbi

Also running into this same issue.

If I comment out my GrpcInstrumentorClient().instrument(), then the errors go away (same stack trace as seen above^). It is also non-deterministic for me unfortunately

matthewgrossman avatar Apr 01 '24 20:04 matthewgrossman