Python 3.8 threading is crashing: `TypeError: run() takes 1 positional argument`

[yardensachs]

Hi all,

We upgraded to Python 3.8 and the new sentry, and we are seeing this error TypeError: run() takes 1 positional argument but 3 were given We are also seeing it showing sometimes as TypeError: run() takes 1 positional argument but 2 were given

We are also using ddtrace.

Exception ignored in thread started by: <bound method Thread._bootstrap of <Thread(AgentWriter, started daemon 140486491760384)>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/threading.py", line 890, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.8/threading.py", line 924, in _bootstrap_inner
Exception ignored in thread started by: <bound method Thread._bootstrap of <Thread(AgentWriter, started daemon 140486500153088)>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/threading.py", line 890, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.8/threading.py", line 924, in _bootstrap_inner
    del _limbo[self]
KeyError: <Thread(AgentWriter, started daemon 140486500153088)>
    del _limbo[self]
KeyError: <Thread(AgentWriter, started daemon 140486500153088)>
Exception in thread AgentWriter:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.8/site-packages/sentry_sdk/integrations/threading.py", line 69, in run
    reraise(*_capture_exception())
  File "/usr/local/lib/python3.8/site-packages/sentry_sdk/_compat.py", line 57, in reraise
    raise value
  File "/usr/local/lib/python3.8/site-packages/sentry_sdk/integrations/threading.py", line 67, in run
    return old_run_func(self, *a, **kw)
  File "/usr/local/lib/python3.8/site-packages/sentry_sdk/integrations/threading.py", line 69, in run
    reraise(*_capture_exception())
  File "/usr/local/lib/python3.8/site-packages/sentry_sdk/_compat.py", line 57, in reraise
    raise value
  File "/usr/local/lib/python3.8/site-packages/sentry_sdk/integrations/threading.py", line 67, in run
    return old_run_func(self, *a, **kw)
  File "/usr/local/lib/python3.8/site-packages/sentry_sdk/integrations/threading.py", line 69, in run
    reraise(*_capture_exception())
  File "/usr/local/lib/python3.8/site-packages/sentry_sdk/_compat.py", line 57, in reraise
    raise value
  File "/usr/local/lib/python3.8/site-packages/sentry_sdk/integrations/threading.py", line 67, in run
    return old_run_func(self, *a, **kw)
TypeError: run() takes 1 positional argument but 3 were given

Does anyone know why this issue is happening?

[untitaker]

Hey, I've never seen this issue before. Can you try downgrading Python to 3.7 or removing ddtrace to rule out inessential factors?

My first thoughts on that stacktrace are that somehow our instrumentation is initialized multiple times. The only way I can imagine this to be possible is if somehow our SDK was initialized while the process is forking.

The exceptions seem to come from threads called AgentWriter so I'm going to look into ddtrace sourcecode to see if they're doing anything funny.

[yardensachs]

@untitaker Thanks! When using py 3.7, this is issue is not happning

[untitaker]

Have you tried removing datadog, and does this happen for other threads as well? I did not find anything suspicious in ddtrace that would explain this

[musterjunk]

https://stackoverflow.com/questions/37116721/python-typeerror-in-threading-function-takes-x-positional-argument-but-y-were

This stackoverflow issue solved it for me. add a comma after your arguments to args after the first one, if you only have one.

[untitaker]

@musterjunk I think this issue is unrelated

[shashwat-nf]

It happens for any unhandled exception. Without any message, I then have to guess the line throwing the exception. Is there any generic way to solve this?

[jcshott]

We're seeing this error in python 3.7.10 & sentry_sdk-1.5.0 any updates on a fix?

I did find this error log that makes me think there is some conflict with ddtrace and sentry. but we utilize both services so would hope to have a solution for them to work together:

Traceback (most recent call last):
  File "/usr/local/lib64/python3.7/site-packages/ddtrace/span.py", line 153, in finish
    self.tracer.record(self._context)
  File "/usr/local/lib64/python3.7/site-packages/ddtrace/tracer.py", line 526, in record
    self.write(trace)
  File "/usr/local/lib64/python3.7/site-packages/ddtrace/tracer.py", line 543, in write
    self.writer.write(spans=spans)
  File "/usr/local/lib64/python3.7/site-packages/ddtrace/internal/writer.py", line 83, in write
    self.start()
  File "/usr/local/lib64/python3.7/site-packages/ddtrace/_worker.py", line 53, in start
    self._thread.start()
  File "/usr/local/lib/python3.7/site-packages/sentry_sdk/integrations/threading.py", line 54, in sentry_start
    return old_start(self, *a, **kw)  # type: ignore
  File "/usr/lib64/python3.7/threading.py", line 848, in start
    raise RuntimeError("threads can only be started once")
RuntimeError: threads can only be started once

[szokeasaurusrex]

@yardensachs @jcshott, I am struggling to reproduce this bug with the information you have provided so far. Can you please provide a specific code snippet which causes this error, and also provide the exact Python version and packages you have installed? And, can you please also verify whether you still observe the bug with the latest SDK version?

[yardensachs]

Yea we don't see this issue anymore FWIW