dspy icon indicating copy to clipboard operation
dspy copied to clipboard

feature(dsp) Add context to details

Open Vasilije1990 opened this issue 4 months ago • 3 comments

Add context to details to make it easier to run evaluations. We want want to make it easier to retrieve context outside of dspy retriever

Vasilije1990 avatar Apr 19 '24 10:04 Vasilije1990

Hi @Vasilije1990 , thanks for the PR. This addition is fine to me, but it seems like in you want to use 'context' within evaluation, and my understanding is this is the 'gold context' from the data, and not actually retrieved context to be used for RAG evaluation. lmk if I'm misinterpreting.

arnavsinghvi11 avatar Apr 28 '24 00:04 arnavsinghvi11

It's a correct estimate. What we are trying to do is to enrich and process gold context, in order to have an enrichment layer. This enrichment layer then evaluates vs qa pairs. In this way, we can see how good our enrichment does the job vs vanilla gold context. Hope that makes sense.

For more details please check here: https://github.com/topoteretes/cognee

Vasilije1990 avatar Apr 30 '24 06:04 Vasilije1990

@Vasilije1990 Can you run ruff check . --fix-only and push again to resolve the failing test? Ready to merge after that!

arnavsinghvi11 avatar May 06 '24 00:05 arnavsinghvi11