Sean MacAvaney

Hi @nirmal2k -- sorry for the delay. If you're looking to reproduce the results in [Training Curricula for Open Domain Answer Re-Ranking](https://arxiv.org/pdf/2004.14269.pdf), I recommend you train from scratch. Instructions are...

> Reranking 1000 documents gives an [email protected]. Is there a reason for the drop? I don't know definitively, but I suspect: - There could be a bias because the training...

Hi @somnath-banerjee, Sorry for the delay. It looks like the vbert model is trying to re-rank based on the "abstract" field (`text_field='abstract'`), whereas only a "text" field is available (`metadata=["docno",...

Yes, so the query-document relevance scores produced by the model are only valuable with respect to other query-document relevance scores. In other words, the only thing that matters is that...

The scores can be positive or negative floating-point values (doesn't need to be in the [0,1] interval).

This fix should take care of the problem! Copies of the qid and docno were not being freed properly. ``` Line # Mem usage Increment Line Contents ================================================ 79 52.750...

@Ricocotam I agree in spirit (and it was originally the plan to have multiple PRs), but there's a lot of interdependence between the changes made (e.g., 3 of the 4...

@tformal I added support for custom thresholds using the same convention as in `trec_eval`. E.g., `{'ndcg_cut.1,15,100'}` will give `ndcg_cut_1`, `ndcg_cut_15`, and `ndcg_cut_100` results. If this sounds good, I'll go ahead...

Do we also want to support a more natural way of requesting specific thresholds? I.e., `{'ndcg_cut_1', 'ndcg_cut_15', 'ndcg_cut_100'}` instead of `{'ndcg_cut.1,15,100'}`? This could be done in a python wrapper that...

@cmacdonald is this what you were looking for? If so, I'll make a PR.