anserini icon indicating copy to clipboard operation
anserini copied to clipboard

Align RunMsMarco with Fatjar regression instructions

Open lintool opened this issue 1 year ago • 2 comments

Today, you can run regressions with the fatjar directly: https://github.com/castorini/anserini/blob/master/docs/fatjar-regressions-v0.35.1.md

Today, you can also do this:

java -cp `ls target/*-fatjar.jar` io.anserini.reproduce.RunMsMarco

These aren't aligned. RunMsMarco only runs a subset of the cases in the fatjar instructions. We should bring the two into alignment - i.e., add the missing cases into RunMsMarco. It probably makes sense to run have then generate the same output to be consistent.

lintool avatar Apr 28 '24 11:04 lintool

Separately:

  • we have fatjar instructions for BEIR, but no io.anserini.reproduce.RunBeir or something similar.
  • we have no repro for V2.1 runs, e.g., add option to io.anserini.reproduce.RunMsMarco.

lintool avatar Apr 28 '24 11:04 lintool

Initial commit #2477 - some errors noted #2480

lintool avatar Apr 30 '24 02:04 lintool

I believe this issue has been resolved.

lintool avatar Aug 08 '24 21:08 lintool