opentelemetry-python-contrib
opentelemetry-python-contrib copied to clipboard
"Failed to detach context" exception for grpc instrumentation
trafficstars
open telemetry version - 0.44b0 python - 3.10 docker versions - ubuntu 20.04 running in Kubernetes cluster, on prem servers
The issue isn't reproduced systematically
many times i get the following exception (everything is fine except for the error printed)
Failed to detach context
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/opentelemetry/trace/__init__.py", line 573, in use_span
yield span
File "/usr/local/lib/python3.10/dist-packages/opentelemetry/sdk/trace/__init__.py", line 1046, in start_as_current_span
yield span
File "/usr/local/lib/python3.10/dist-packages/opentelemetry/instrumentation/grpc/_client.py", line 211, in _intercept_server_stream
yield from invoker(request_or_iterator, metadata)
GeneratorExit
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/opentelemetry/context/__init__.py", line 163, in detach
_RUNTIME_CONTEXT.detach(token) # type: ignore
File "/usr/local/lib/python3.10/dist-packages/opentelemetry/context/contextvars_context.py", line 50, in detach
self._current_context.reset(token) # type: ignore
ValueError: <Token var=<ContextVar name='current_context' default={} at 0x7fbf8d8b2bb0> at 0x7fbf4c4b9d40> was created in a different Context
there are related bugs but not regarding grpc instrumentation https://github.com/open-telemetry/opentelemetry-python-contrib/issues/904 https://github.com/open-telemetry/opentelemetry-python/issues/2606
Also running into this same issue.
If I comment out my GrpcInstrumentorClient().instrument(), then the errors go away (same stack trace as seen above^). It is also non-deterministic for me unfortunately