[Bug]: Langfuse Integration Error: TypeError: cannot pickle '_thread.RLock' object

[bgeneto]

What happened

When using the Langfuse integration with litellm.log_langfuse_v2, a TypeError: cannot pickle '_thread.RLock' object is raised during the logging process. This appears to be caused by attempting to deepcopy data containing a _thread.RLock object, which is not serializable.

To Reproduce

While the exact steps to reproduce may vary depending on the specific data being logged, the general pattern involves:

Using success_callback: [langfuse] to log data to Langfuse. The data being logged (directly or within nested structures) contains a _thread.RLock object.

Expected behavior

The Langfuse integration should successfully log data without raising a TypeError. Ideally, the integration should handle non-serializable objects gracefully.

Environment

Python version: v3.11.8 litellm version: 1.49.6 (from 2024-10-17) langfuse version: v2.85.1 OSS

Additional context

It seems that this issue was introduced very recently. The error traceback points to the deepcopy operation within the _log_langfuse_v2 function as the source of the issue. This suggests that the integration attempts to create a deep copy of the data being logged, which fails due to the presence of the non-serializable _thread.RLock object.

Maybe a similar fix as in #3384 can be applied?

Relevant log output

{"message": "Langfuse Layer Error - Traceback (most recent call last):\n File "/usr/local/lib/python3.11/site-packages/litellm/integrations/langfuse.py", line 373, in _log_langfuse_v2\n optional_params.pop("metadata")\nKeyError: 'metadata'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/usr/local/lib/python3.11/site-packages/litellm/integrations/langfuse.py", line 387, in _log_langfuse_v2\n new_metadata[key] = copy.deepcopy(value)\n ^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 146, in deepcopy\n y = copier(x, memo)\n ^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 206, in _deepcopy_list\n append(deepcopy(a, memo))\n ^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 146, in deepcopy\n y = copier(x, memo)\n ^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 231, in _deepcopy_dict\n y[deepcopy(key, memo)] = deepcopy(value, memo)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 172, in deepcopy\n y = _reconstruct(x, memo, *rv)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 271, in _reconstruct\n state = deepcopy(state, memo)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 146, in deepcopy\n y = copier(x, memo)\n ^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 231, in _deepcopy_dict\n y[deepcopy(key, memo)] = deepcopy(value, memo)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 172, in deepcopy\n y = _reconstruct(x, memo, *rv)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 271, in _reconstruct\n state = deepcopy(state, memo)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 146, in deepcopy\n y = copier(x, memo)\n ^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 231, in _deepcopy_dict\n y[deepcopy(key, memo)] = deepcopy(value, memo)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 146, in deepcopy\n y = copier(x, memo)\n ^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 231, in _deepcopy_dict\n y[deepcopy(key, memo)] = deepcopy(value, memo)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 172, in deepcopy\n y = _reconstruct(x, memo, *rv)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 271, in _reconstruct\n state = deepcopy(state, memo)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 146, in deepcopy\n y = copier(x, memo)\n ^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 231, in _deepcopy_dict\n y[deepcopy(key, memo)] = deepcopy(value, memo)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 172, in deepcopy\n y = _reconstruct(x, memo, *rv)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 271, in _reconstruct\n state = deepcopy(state, memo)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 146, in deepcopy\n y = copier(x, memo)\n ^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 231, in _deepcopy_dict\n y[deepcopy(key, memo)] = deepcopy(value, memo)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 172, in deepcopy\n y = _reconstruct(x, memo, *rv)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 271, in _reconstruct\n state = deepcopy(state, memo)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 146, in deepcopy\n y = copier(x, memo)\n ^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 231, in _deepcopy_dict\n y[deepcopy(key, memo)] = deepcopy(value, memo)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 172, in deepcopy\n y = _reconstruct(x, memo, *rv)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 271, in _reconstruct\n state = deepcopy(state, memo)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 146, in deepcopy\n y = copier(x, memo)\n ^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 231, in _deepcopy_dict\n y[deepcopy(key, memo)] = deepcopy(value, memo)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/copy.py", line 161, in deepcopy\n rv = reductor(4)\n ^^^^^^^^^^^\nTypeError: cannot pickle '_thread.RLock' object\n", "level": "ERROR", "timestamp": "2024-10-17T16:37:06.593106"}

Twitter / LinkedIn details

No response

[bgeneto]

@krrishdholakia Just to let you know that using the recently added model jina_ai/jina-embeddings-v3 triggers this langfuse error! I mean, the embedding works great (thx!) and the litellm-aembedding trace appears in langfuse UI. Problem is any subsequent query using the document (litellm-acompletion) triggers this error. (I'm using open-webui to test).

[krrishdholakia]

can you share what the first + subsequent queries look like? @bgeneto

[tcapelle]

Same error here:

Exception has occurred: InstructorRetryException
cannot pickle '_thread.RLock' object
TypeError: cannot pickle '_thread.RLock' object

The above exception was the direct cause of the following exception:

tenacity.RetryError: RetryError[<Future at 0x12d8d6060 state=finished raised TypeError>]

The above exception was the direct cause of the following exception:

  File "/Users/tcapelle/work/evalForge/evalforge/forge.py", line 299, in get_task_description
    response = llm_client.chat.completions.create(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tcapelle/work/evalForge/evalforge/forge.py", line 450, in predict
    llm_task_description = self.get_task_description(data)

Had to remove litllm from my. codebase

[ghost]

@bgeneto @krrishdholakia I've just created a PR #6563 that fixes this issue. Feel free to review and merge 🙂

[ghost]

@krrishdholakia @ishaan-jaff sorry for tagging you, but could you take a look at it? It appears to be happening in almost every request, so integration is basically broken now.

I had a research but still have no idea how Lock object passes to metadata, but at least it shouldn't block integration

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[ishaan-jaff]

the issue is fixed on latest correct ? @sevkar @bgeneto @tcapelle

[bgeneto]

the issue is fixed on latest correct ? @sevkar @bgeneto @tcapelle

It hasn't happened to me in a while...

[ghost]

the issue is fixed on latest correct ? @sevkar @bgeneto @tcapelle

Yes. Was merged as part of #6590 3 months ago