deepeval
deepeval copied to clipboard
confident-ai
The LLM Evaluation Framework
**❗BEFORE YOU BEGIN❗** **Is your feature request related to a problem? Please describe.** I'm embedding your `evaluate` function in a program and it outputs stuff like this: ✅ Tests finished!...
**Describe the bug** Cache can get corrupted and block execution. **To Reproduce** `touch .temp-deepeval-cache.json` Then run a program with caching turned on. The bug can also be sort of seen...
Hi! I have noticed that G-Eval uses .generate_raw_response for its calculations. I wanted to understand the inner workings of the metric and thus tried to dive deeper into the code....
As discussed on Discord, we need to know what prompts you are serving the evaluation LLM. https://hamel.dev/blog/posts/prompt/ I need to see the prompt to help debug when the framework fails...
**Describe the bug** If you evaluate twice, with cache turned on, the result loaded from cache will not have a score. **To Reproduce** ```python from deepeval import evaluate from deepeval.metrics...
The documentation for the `evaluate` function does not say what it returns.
**Describe the bug** My definition of a hallucination is a fact that occurs in the output which is not in the context. As the docs say, a "contradicted context". But...
**Describe the bug** I get a disk error when trying to run a test. Might be a bug in how some temp file is being cached by the DeepEval library...
**Is your feature request related to a problem? Please describe.** Using deepeval inside the docker web app and not cli so don't want it to write anything to the application...
Integrated chromadb to synthesizer (context_generator) to save embeddings for documents that have already been supplied to hidden folder. Future Consideration - [ ] Use search from chromaDB directly maybe faster...
