pytrec_eval
pytrec_eval copied to clipboard
Using pytrec_eval.RelevanceEvaluator, do we sort the results internally or use as it is provides?
I am using
evaluator = pytrec_eval.RelevanceEvaluator(qrels, {map_string, ndcg_string, recall_string, precision_string})
scores = evaluator.evaluate(results)
Now result is
{qid: {docid: embeddding_score, docid: embeddding_score} }
so we sort the results using scores before using for metrics calculation? I am assuming it has to be as dict will not have fix indexing.
can we avoid this sorting? in that case do I have to manipulate the scores manually to achieve desired results?