pygaggle icon indicating copy to clipboard operation
pygaggle copied to clipboard

Append to MS MARCO replication log and fix spacy error

Open jx3yang opened this issue 2 years ago • 1 comments

Description

Replicate the MS MARCO document retrieval results.

Issue Encountered

The spacy errors mentioned in #267. This PR suggests a fix for those issues.

Experiment Results Replication

The mrr's I got differ slightly in the 5th decimal place:

fh image The expected mrr is 0.38882, but got 0.3888.

sh image The expected mrr is 0.33617 but got 0.33614.

Environment

Python 3.7.13 GPU: Tesla P100 (Google Colab)

See the following Colab notebooks:

jx3yang avatar May 23 '22 15:05 jx3yang

Hi all, just bumping this PR, I think it is OK to merge. I was able to replicate the revised results exactly, using the new changes proposed in this revision. Feel free to add a replication entry on my behalf, I used a GeForce RTX 2080 Ti on a Ubuntu 21.10 machine. Thanks

joelrorseth avatar Aug 05 '22 23:08 joelrorseth