RocketQA icon indicating copy to clipboard operation
RocketQA copied to clipboard

Could I arise one PR to fix MRR metric statics method ???

Open Yazooliu opened this issue 2 years ago • 1 comments

I saw https://github.com/PaddlePaddle/RocketQA/blob/main/research/DuReader-Retrieval-Baseline/metric/evaluation.py line 135 is wrong. 百度MRR代码问题截图

for MRR 定义: MRR定义

计算分母时,分母表示的是问题对应的推理答案长度。

而您的代码在计算时分母是ground_truth的长度。实际应用中ground_truth的长度和推理结果的长度不一定相同,所以您的代码有问题,请求提交PR修改,请批准。

祝好 Yazooliu

Yazooliu avatar Dec 01 '23 09:12 Yazooliu

Any respond?Could I try?

Yazooliu avatar Dec 19 '23 07:12 Yazooliu