django-polymorphic icon indicating copy to clipboard operation
django-polymorphic copied to clipboard

TypeError: cannot pickle '_thread.lock' object

Open rrauenza opened this issue 1 year ago • 2 comments

I'll start with I don't know WHY the Q object has a _thread.lock object in it ... but ...

I think this is caused by this code, and the deepcopy of the Q object:

translate_polymorphic_Q_object(queryset_model, copy.deepcopy(q), using=using) for q in args

~~I wonder if the code should be using q.clone() (undocumented?) or even better, q.all() (which does the same as q.clone()), which supposedly makes a copy of the query?~~ Oops, a Q object is not a query. It's a Q().

I noticed this seems similar to issue https://github.com/django-polymorphic/django-polymorphic/issues/524 that originates at the same line of code.

rrauenza avatar May 09 '23 19:05 rrauenza

I have a proposed PR that has a custom deepcopy to avoid these pickle issues.

rrauenza avatar May 09 '23 21:05 rrauenza

full stack trace, for posterity. It isn't clear how this _thread object appears in the deepcopy...

  File ".../.venv/lib/python3.11/site-packages/polymorphic/query_translate.py", line 95, in translate_polymorphic_filter_definitions_in_args
    return [
           ^
  File ".../.venv/lib/python3.11/site-packages/polymorphic/query_translate.py", line 96, in <listcomp>
    translate_polymorphic_Q_object(queryset_model, copy.deepcopy(q), using=using) for q in args
                                                   ^^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 153, in deepcopy
    y = copier(memo)
        ^^^^^^^^^^^^
  File ".../.venv/lib/python3.11/site-packages/django/utils/tree.py", line 53, in __deepcopy__
    obj.children = copy.deepcopy(self.children, memodict)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 146, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 206, in _deepcopy_list
    append(deepcopy(a, memo))
           ^^^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 146, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 211, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 211, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
         ^^^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 146, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 206, in _deepcopy_list
    append(deepcopy(a, memo))
           ^^^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 146, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 146, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 146, in deepcopy
    y = copier(x, memo)
        ^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python-3.11.3/lib/python3.11/copy.py", line 161, in deepcopy
    rv = reductor(4)
         ^^^^^^^^^^^
TypeError: cannot pickle '_thread.lock' object

rrauenza avatar May 09 '23 23:05 rrauenza