pygaggle
pygaggle copied to clipboard
Append to MS MARCO replication log and fix spacy error
Description
Replicate the MS MARCO document retrieval results.
Issue Encountered
The spacy errors mentioned in #267. This PR suggests a fix for those issues.
Experiment Results Replication
The mrr
's I got differ slightly in the 5th decimal place:
fh
The expected
mrr
is 0.38882, but got 0.3888.
sh
The expected
mrr
is 0.33617 but got 0.33614.
Environment
Python 3.7.13 GPU: Tesla P100 (Google Colab)
See the following Colab notebooks:
Hi all, just bumping this PR, I think it is OK to merge. I was able to replicate the revised results exactly, using the new changes proposed in this revision. Feel free to add a replication entry on my behalf, I used a GeForce RTX 2080 Ti on a Ubuntu 21.10 machine. Thanks