baidu_ultr_dataset
baidu_ultr_dataset copied to clipboard
ChuXiaokai
Document Order in Test Dataset
Thank you very much for making this dataset public. I have a quick question about the test dataset. I understand that for each query you take the top30 documents from the Search engine + another ~30 from the top 1000. When these results are presented to the experts, is the order shuffled? 0r are they shown in their original order? Thx for your help
The ranking position is not presented to the experts.
2023年3月19日 18:49，citizenkeynes @.***> 写道：
Thank you very much for making this dataset public. I have a quick question about the test dataset. I understand that for each query you take the top30 documents from the Search engine + another ~30 from the top 1000. When these results are presented to the experts, is the order shuffled? 0r are they shown in their original order? Thx for your help
So when the experts judge the documents they are presented in no particular order?
They judge one document every time and do not compare with others.
2023年3月19日 19:08，citizenkeynes @.***> 写道：
So when they judge the documents they are presented in no particular order?
— Reply to this email directly, view it on GitHub https://github.com/ChuXiaokai/baidu_ultr_dataset/issues/13#issuecomment-1475212618, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPSGYGLGEDUIZ7ODVMOAKLW43SLLANCNFSM6AAAAAAWABFNN4. You are receiving this because you commented.
👍 Thanks for clarifying
On Sun, 19 Mar 2023, 12:29 pm zoulixin93, @.***> wrote:
They judge one document every time and do not compare with others.
2023年3月19日 19:08，citizenkeynes @.***> 写道：
So when they judge the documents they are presented in no particular order?
— Reply to this email directly, view it on GitHub < https://github.com/ChuXiaokai/baidu_ultr_dataset/issues/13#issuecomment-1475212618>, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AHPSGYGLGEDUIZ7ODVMOAKLW43SLLANCNFSM6AAAAAAWABFNN4 . You are receiving this because you commented.
— Reply to this email directly, view it on GitHub https://github.com/ChuXiaokai/baidu_ultr_dataset/issues/13#issuecomment-1475218541, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHX4DJ4OOXBRVG6OEHM7Z43W43UZXANCNFSM6AAAAAAWABFNN4 . You are receiving this because you authored the thread.Message ID: @.***>