Jimmy Lin

Thoughts about API design

17

Just thoughts... @ronakice @sahel-sh reactions welcome. Two methods: `rerank` and `rerank_batch`. Method signature: ``` def rerank(self, query: Query, candidates: Candidates, k: int = 10) -> Candidates: ``` Commentary: + I...

Thoughts about design philosophy of RankLLM

6

What is RankLLM? I can think of two obvious answers: **Approach 1. RankLLM is a fully-integrated layer on top of Anserini and Pyserini.** If this is the case, then we...

Refactor tolerance settings for MS MARCO dense vector regressions

1

Continuation of #2538 + tweak docs for flat indexes. + refactor tolerance values for HNSW indexes, calibrate wrt flat index scores.

Discussion: REST API routes for different corpus/model combinations - how do we name?

7

We have different corpus/model combinations. For example, we have `msmarco-v1-passage`, `msmarco-v1-passage.splade-pp-ed`, and `msmarco-v1-passage.cos-dpr-distil`. They are all on MS MARCO V1 Passage, but different models - the first for BM25, SPLADE,...

https://x.com/EmilevanKrieken/status/1801267652565893350 https://x.com/lintool/status/1788569556157366631 https://x.com/maliannejadi/status/1788849915457581384

What to do when there's contemporaneous work? tl;dr - you're screwed either way... Reference them: + get criticized for not being novel + get criticized if your results are weaker...

Reminder to self.

The current implementation of fusion is built around `TrecRun`, e.g., https://github.com/castorini/anserini/blob/master/src/main/java/io/anserini/fusion/TrecRunFuser.java But we already have `ScoredDocs`: https://github.com/castorini/anserini/blob/master/src/main/java/io/anserini/search/ScoredDocs.java Should we refactor our implementation to use `ScoredDocs`? Potential cons: + faster? +...