dd-trace-py icon indicating copy to clipboard operation
dd-trace-py copied to clipboard

aioredis raises CancelledError in _finish_span

Open mabdinur opened this issue 3 years ago • 2 comments

Which version of dd-trace-py are you using?

~~0.53.0~~ 0.58.0

Which version of pip are you using?

21.3.1

Which version of the libraries are you using?

django == 3.2.11 django-redis = 5.0.0 channels == 3.0.4 channels_redis = 5.0.0 daphne == 3.0.2

How can we reproduce your problem?

I am using code similar to the following:

asgi.py

import django
from channels.routing import get_default_application
from ddtrace.contrib.asgi import TraceMiddleware

django.setup()
application = TraceMiddleware(get_default_application())

routing.py

from django.urls import re_path
import my_app.consumers

websocket_urlpatterns = [
    re_path(r"^ws/test/$", consumers.TestConsumer.as_asgi()),
]

my_app/consumers.py

from channels.generic.websocket import WebsocketConsumer

class TestConsumer(WebsocketConsumer):
    groups = ["broadcast"]

    def connect(self):
        self.accept()

    def receive(self, text_data=None, bytes_data=None):
        raise Exception("An test exception")

I am running the application with: ddtrace-run daphne asgi:application --bind 0.0.0.0 --port 8001

What is the result that you get?

I don't get any traces at all, and my logs show this:

handle: <Handle traced_13_execute_command.<locals>._finish_span(<Future cancelled>) at /usr/local/lib/python3.10/site-packages/ddtrace/contrib/aioredis/patch.py:140>

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/local/lib/python3.10/site-packages/ddtrace/contrib/aioredis/patch.py", line 146, in _finish_span
    future.result()
asyncio.exceptions.CancelledError

What is the result that you expected?

No errors

mabdinur avatar Feb 11 '22 17:02 mabdinur

@mabdinur Do you know what was the last working version?

yuri1992 avatar Mar 01 '22 07:03 yuri1992

We are seeing the same issue with ddtrace-py 1.1.3 and a very similar stack (django + redis + channels). I will upgrade to the latest 1.2.2 tomorrow to see if that resolves the issue and report back.

marknotfound avatar Jul 06 '22 00:07 marknotfound

Getting this on the latest ddtrace myself (1.4.0). It's not clear if the issue is due to this package, however, has anyone figured out what is the cause of this?

jacktreg avatar Aug 22 '22 22:08 jacktreg

Same here w/ ddtrace 1.4.1 and a similar stack.

adambyer avatar Aug 24 '22 19:08 adambyer

@adambyer @jacktreg @marknotfound I'm currently trying to reproduce this error with limited success. Could you share any more details that I could use to reproduce this including:

  • Python version
  • ddtrace version
  • django, redis, channels versions
  • Full error/traceback logs (the current log above only shows that there was a CancelledError, I cannot tell yet whether that is due to ddtrace or aioredis) Thanks!

Yun-Kim avatar Sep 14 '22 19:09 Yun-Kim