trulens
trulens copied to clipboard
[FEAT] Add python 3.12 & 3.13. support for OpenAI
Feature Description python 3.12 - 3.13. support
Reason Users are wanting to use TruLens on python 3.13
Importance of Feature What value does this feature unlock for you?
3.12 should work already I believe. Are they running into some issue there?
@sfc-gh-dkurokawa I reproduced this same issue on both 3.12 and 3.13: https://github.com/truera/trulens/issues/1987
tagging related fix: https://github.com/truera/trulens/pull/2005
I manually applied the change
diff .venv/lib/python3.10/site-packages/trulens/core/utils/pyschema.py .venv/lib/python3.10/site-packages/trulens/core/utils/pyschema.py.bak
64,66c64,66
< Try to get the attribute `k` of the given object. If it’s a @property or
< functools.cached_property (including OpenAI’s cached_property shim), this
< will invoke the descriptor to compute and return the value.
---
> Try to get the attribute `k` of the given object. This may evaluate some
> code if the attribute is a property and may fail. In that case, an dict
> indicating so is returned.
68,69c68,69
< If `get_prop` is False, will raise ValueError for any property/descriptor.
< On exceptions, returns a dict with an ERROR key.
---
> If `get_prop` is False, will not return contents of properties (will raise
> `ValueException`).
71c71
< # Fetch the descriptor or raw attribute without invoking it
---
>
74,80c74,85
< # Detect both plain @property and stdlib cached_property
< is_prop = isinstance(v, property)
< is_std_cached = isinstance(v, functools.cached_property)
<
< if is_prop or is_std_cached:
< if not get_prop:
< raise ValueError(f"{k} is a property/descriptor")
---
> is_prop = False
> try:
> # OpenAI version 1 classes may cause this isinstance test to raise an
> # exception.
> is_prop = isinstance(v, property)
> except Exception as e:
> return {constant_utils.ERROR: Obj.of_object(e)}
>
> if is_prop:
> if not get_prop:
> raise ValueError(f"{k} is a property")
>
82,83c87,89
< # Invoke the descriptor protocol to compute (and cache) the value
< return v.__get__(obj, type(obj))
---
> v = v.fget(obj)
> return v
>
86,88c92,93
<
< # Not a property/descriptor we care about—return the raw value
< return v
---
> else:
> return v
The problem still exist with the fix
sqlite> select * from trulens_feedbacks;
feedback_result_hash_aa220aa3e518159076e2194b8d6564c8|record_hash_0b4b8eed912e632b73e92ca7d2841850|feedback_definition_hash_4eddfa5e9b512787b9be00e5f0048ed9|1748927718.44679|failed|Traceback (most recent call last):
File "/Users/woody.yang/learn_llm/azure-ai-search-python-playground/.venv/lib/python3.10/site-packages/trulens/core/feedback/feedback.py", line 949, in run
core_endpoint.Endpoint.track_all_costs_tally(
File "/Users/woody.yang/learn_llm/azure-ai-search-python-playground/.venv/lib/python3.10/site-packages/trulens/core/feedback/endpoint.py", line 570, in track_all_costs_tally
result, cbs = Endpoint.track_all_costs(
File "/Users/woody.yang/learn_llm/azure-ai-search-python-playground/.venv/lib/python3.10/site-packages/trulens/core/feedback/endpoint.py", line 543, in track_all_costs
return Endpoint._track_costs(
File "/Users/woody.yang/learn_llm/azure-ai-search-python-playground/.venv/lib/python3.10/site-packages/trulens/core/feedback/endpoint.py", line 647, in _track_costs
result: T = __func(*args, **kwargs)
File "/Users/woody.yang/learn_llm/azure-ai-search-python-playground/.venv/lib/python3.10/site-packages/trulens/core/feedback/feedback.py", line 507, in __call__
return self.imp(*args, **valid_kwargs)
File "/Users/woody.yang/learn_llm/azure-ai-search-python-playground/.venv/lib/python3.10/site-packages/trulens/feedback/llm_provider.py", line 384, in context_relevance_with_cot_reasons
return self.generate_score_and_reasons(
File "/Users/woody.yang/learn_llm/azure-ai-search-python-playground/.venv/lib/python3.10/site-packages/trulens/feedback/llm_provider.py", line 155, in generate_score_and_reasons
response = self.endpoint.run_in_pace(
File "/Users/woody.yang/learn_llm/azure-ai-search-python-playground/.venv/lib/python3.10/site-packages/trulens/core/feedback/endpoint.py", line 336, in run_in_pace
raise RuntimeError(
RuntimeError: Endpoint OpenAIEndpoint request failed 4 time(s):
'cached_property' object has no attribute 'completions'
'cached_property' object has no attribute 'completions'
'cached_property' object has no attribute 'completions'
'cached_property' object has no attribute 'completions'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/woody.yang/learn_llm/azure-ai-search-python-playground/.venv/lib/python3.10/site-packages/trulens/core/feedback/feedback.py", line 964, in run
raise RuntimeError(
RuntimeError: Evaluation of context_relevance_with_cot_reasons failed on inputs:
{'context': 'Overview \n'
'Introducing PerksPlus - the ultimate benefits program designed to '
'support .
|{"calls": []}||context_relevance_with_cot_reasons|{"n_requests": 0, "n_successful_requests": 0, "n_completion_requests": 0, "n_classification_requests": 0, "n_classes": 0, "n_embedding_requests": 0, "n_embeddings": 0, "n_tokens": 0, "n_stream_chunks": 0, "n_prompt_tokens": 0, "n_completion_tokens": 0, "n_cortex_guardrails_tokens": 0, "cost": 0.0, "cost_currency": "USD"}|