pytrec_eval icon indicating copy to clipboard operation
pytrec_eval copied to clipboard

Using pytrec_eval.RelevanceEvaluator, do we sort the results internally or use as it is provides?

Open SDcodehub opened this issue 7 months ago • 1 comments

I am using

        evaluator = pytrec_eval.RelevanceEvaluator(qrels, {map_string, ndcg_string, recall_string, precision_string})
        scores = evaluator.evaluate(results)

Now result is {qid: {docid: embeddding_score, docid: embeddding_score} }

so we sort the results using scores before using for metrics calculation? I am assuming it has to be as dict will not have fix indexing.

can we avoid this sorting? in that case do I have to manipulate the scores manually to achieve desired results?

SDcodehub avatar Jul 19 '24 13:07 SDcodehub