trectools icon indicating copy to clipboard operation
trectools copied to clipboard

Fixed nan returned for ndcg, rprec if no relevant docs retrieved

Open TSoli opened this issue 2 months ago • 0 comments

I noticed that if no relevant documents are retrieved for a run then NDCG, Rprec and recip_rank all returned nan. For NDCG and Rprec I believe these should be 0 instead. I left recip_rank since it is not obvious which would be preferred here.

I wrote some tests for the fixes I made which ensure that the relevant metrics are 0 if no relevant documents were retrieved or if the documents retrieved were all unlabelled.

Note this is different from #25 since there are relevant documents they were just not retrieved during the run (or up to the depth evaluated). With regards to #25 it seems that the entries for NDCG per query do not exist if there are not relevant documents in the collection. This is not fixed here.

TSoli avatar Dec 11 '24 04:12 TSoli