llama_index icon indicating copy to clipboard operation
llama_index copied to clipboard

[Bug]: CohereRerankRelevancyMetric relevance_score is not used correctly

Open olyashok opened this issue 11 months ago • 1 comments

Bug Description

compute function should be relevance_scores = [r.relevance_score for r in results.results] instead of relevance_scores = [r.relevance_score for r in results]

since rerank returns id,results,meta tuple

Version

0.10.20

Steps to Reproduce

run https://docs.llamaindex.ai/en/stable/examples/evaluation/retrieval/retriever_eval/

Relevant Logs/Tracbacks

No response

olyashok avatar Mar 26 '24 19:03 olyashok

It's great to see you've identified a potential bug and proposed a solution! Your contribution would be valuable to the project. Feel free to create a pull request with your proposed changes so that the maintainers can review and discuss them. Keep up the good work!

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

dosubot[bot] avatar Mar 26 '24 19:03 dosubot[bot]