[Bug]: Calling opik.search_traces sometimes returns an error

[jverre]

What component(s) are affected?

• [ ] Python SDK
• [ ] Opik UI
• [ ] Opik Server
• [ ] Documentation

Opik version

• Opik version: latest

Describe the problem

I'm running:

import opik
client = opik.Opik()

client.search_traces(project_name="Default Project")

and sometimes get the error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jacquesverre/Documents/code/opik/sdks/python/src/opik/api_objects/opik_client.py", line 629, in search_traces
    page_traces = self._rest_client.traces.get_traces_by_project(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/py312_llm_eval/lib/python3.12/site-packages/tenacity/__init__.py", line 336, in wrapped_f
    return copy(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/py312_llm_eval/lib/python3.12/site-packages/tenacity/__init__.py", line 475, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/py312_llm_eval/lib/python3.12/site-packages/tenacity/__init__.py", line 376, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/py312_llm_eval/lib/python3.12/site-packages/tenacity/__init__.py", line 398, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/py312_llm_eval/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/py312_llm_eval/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/homebrew/Caskroom/miniconda/base/envs/py312_llm_eval/lib/python3.12/site-packages/tenacity/__init__.py", line 478, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/Users/jacquesverre/Documents/code/opik/sdks/python/src/opik/rest_api/traces/client.py", line 182, in get_traces_by_project
    parse_obj_as(
  File "/Users/jacquesverre/Documents/code/opik/sdks/python/src/opik/rest_api/core/pydantic_utilities.py", line 65, in parse_obj_as
    return adapter.validate_python(dealiased_object)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/py312_llm_eval/lib/python3.12/site-packages/pydantic/type_adapter.py", line 135, in wrapped
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/py312_llm_eval/lib/python3.12/site-packages/pydantic/type_adapter.py", line 366, in validate_python
    return self.validator.validate_python(object, strict=strict, from_attributes=from_attributes, context=context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 16 validation errors for TracePagePublic
content.0.input
  Input should be a valid dictionary [type=dict_type, input_value=[{'content': '\n        Y...     ', 'role': 'user'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/dict_type
content.1.input
  Input should be a valid dictionary [type=dict_type, input_value=[{'content': '\n        Y...     ', 'role': 'user'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/dict_type
content.2.input
  Input should be a valid dictionary [type=dict_type, input_value=[{'content': '\n        Y...     ', 'role': 'user'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/dict_type
content.3.input
  Input should be a valid dictionary [type=dict_type, input_value=[{'content': '\n        Y...     ', 'role': 'user'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/dict_type
content.4.input
  Input should be a valid dictionary [type=dict_type, input_value=[{'content': '\n        Y...     ', 'role': 'user'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/dict_type
content.5.input
  Input should be a valid dictionary [type=dict_type, input_value=[{'content': '\n        Y...     ', 'role': 'user'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/dict_type
content.6.input
  Input should be a valid dictionary [type=dict_type, input_value=[{'content': '\n        Y...     ', 'role': 'user'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/dict_type
content.7.input
  Input should be a valid dictionary [type=dict_type, input_value=[{'role': 'user', 'conten...on of LLMs important?'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/dict_type
content.8.input
  Input should be a valid dictionary [type=dict_type, input_value=[{'role': 'user', 'conten...on of LLMs important?'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/dict_type
content.9.input
  Input should be a valid dictionary [type=dict_type, input_value=[{'role': 'user', 'conten...on of LLMs important?'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/dict_type
content.10.input
  Input should be a valid dictionary [type=dict_type, input_value=[{'role': 'user', 'conten...on of LLMs important?'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/dict_type
content.11.input
  Input should be a valid dictionary [type=dict_type, input_value=[{'role': 'user', 'conten...on of LLMs important?'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/dict_type
content.12.input
  Input should be a valid dictionary [type=dict_type, input_value=[{'role': 'user', 'conten...on of LLMs important?'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/dict_type
content.13.input
  Input should be a valid dictionary [type=dict_type, input_value=[{'role': 'user', 'conten...on of LLMs important?'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/dict_type
content.14.input
  Input should be a valid dictionary [type=dict_type, input_value=[{'role': 'user', 'conten...on of LLMs important?'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/dict_type
content.15.input
  Input should be a valid dictionary [type=dict_type, input_value=[{'role': 'user', 'conten...on of LLMs important?'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.9/v/dict_type

This seems to depend on the project, it doesn't always happen

Reproduction steps

No response

[dsblank]

@jverre does this still happen?

[jverre]

Yes, it's due to data inconsistencies between expected data formats and what you can send. It's planned as OPIK-794 to be tackled in the upcoming week

[Priya9789]

pydantic models support in opik ? im using openai_client.beta.chat.completions.parse() this but return for example like response_format=TranscribedOfflineResponse, pydantic model how can i proceed? model: gpt-4o-2024-08-06 response_format: <class 'teacher_app.views.offline_ai_evaluation.TranscribedOfflineResponse'> created_from: openai type: openai_chat

[andrescrz]

Hi all,

I merged a fix for this:

• https://github.com/comet-ml/opik/pull/2159

I'll close this issue once it's verified that it no longer reproduces again. Regards

[andrescrz]

Hi all,

The fix has been released as part of Opik 1.7.24:

• https://github.com/comet-ml/opik/releases/tag/1.7.24

Closing this issue so far. Please reach out again or re-open if you encounter it again. Regards