ClariQ icon indicating copy to clipboard operation
ClariQ copied to clipboard

How do you get <relevance_score> in train.qrel

Open ShiyuNee opened this issue 1 year ago • 2 comments

I'd like to know that why the relevance score can be 0 and i notice that there are only 628 facets that have documents whose relevance score is larger than 0. However, there are 1070 facets in train.qrel.

ShiyuNee avatar Apr 02 '23 14:04 ShiyuNee

There are 10 facets with no relevant documents in train.tsv. The facets are: F0322 F0155 F0140 F0349 F0041 F0753 F0365 F0276 F0166 F0294

ShiyuNee avatar Apr 03 '23 02:04 ShiyuNee

We have adapted the qrel files of the Web track where a relevance score of "0" means that the document was assessed but judged as irrelevant. The difference with other irrelevant documents is that they are not assessed. That's true, there are some facets with no relevant documents.

aliannejadi avatar Apr 13 '23 16:04 aliannejadi