deepeval
deepeval copied to clipboard
Corrupted cache can make it impossible to run deepeval
Describe the bug Cache can get corrupted and block execution.
To Reproduce
touch .temp-deepeval-cache.json
Then run a program with caching turned on.
The bug can also be sort of seen in the code.
The function signature for get_cached_test_run
is
def get_cached_test_run(
self, from_temp: bool = False
) -> Union[CachedTestRun, None]:
And yet the way it is used is:
cached_test_run = self.get_cached_test_run(from_temp=to_temp)
cached_test_run.test_cases_lookup_map[test_case_cache_key] = (
new_cache_test_case
)
There is no handling for the None case, which results in an exception.
File "/Users/paul.prescod/.venvs/evalu8/lib/python3.10/site-packages/deepeval/evaluate.py", line 348, in a_execute_test_cases
test_run_cache_manager.cache_test_case(
File "/Users/paul.prescod/.venvs/evalu8/lib/python3.10/site-packages/deepeval/test_run/cache.py", line 136, in cache_test_case
cached_test_run.test_cases_lookup_map[test_case_cache_key] = (
AttributeError: 'NoneType' object has no attribute 'test_cases_lookup_map'
https://github.com/confident-ai/deepeval/blob/004e1cb42d11d9de039a0db047423f9576f70800/deepeval/test_run/cache.py#L136
If cached_test_run
can be None, the code should handle that.
Expected behavior Replace the cache if it is missing or corrupted.
Deepeval version 0.21.42