haystack-core-integrations icon indicating copy to clipboard operation
haystack-core-integrations copied to clipboard

Langfuse: Create a simple way to add evaluation pipeline trace to trace of pipeline to get results

Open mrm1001 opened this issue 7 months ago • 0 comments

Is your feature request related to a problem? Please describe.

At the moment, users need to run 2 pipelines to get evaluation metrics: the pipeline that iterates over the documents to get retrieved contexts and predicted answers, and a second pipeline to get the evaluation metrics. All the metrics in the library have individual, as well as aggregated scores. It would be great if there could be a simple way to visualise the trace for the specific RAG query alongside the evaluation pipeline trace for that query.

Describe the solution you'd like

So for example, I would like to have the trace for a specific RAG query together with the trace for the ExactAnswerMatch evaluator run on the same query/answer.

mrm1001 avatar Jul 10 '24 13:07 mrm1001