CodeSearchNet icon indicating copy to clipboard operation
CodeSearchNet copied to clipboard
verified publisher icon github

NDCG computation

Open Zamony opened this issue 5 years ago • 1 comments

Hi there! In the function corresponding to NDCG calculation I noticed that ignore_rank_of_non_annotated_urls flag is present. https://github.com/github/CodeSearchNet/blob/76a006fda591591f196fd0aef1d6282af3135f71/src/relevanceeval.py#L76

The question is: are your results in the paper calculated with ignore_rank_of_non_annotated_urls=True?

Zamony avatar Feb 27 '21 18:02 Zamony

Both are reported. In Table 4, NDCG-Within for True, NDCG-All for False.

Alex-HaochenLi avatar Sep 10 '22 07:09 Alex-HaochenLi