rank_llm
rank_llm copied to clipboard
castorini
Repository for prompt-decoding using LLMs (GPT3.5, GPT4, Vicuna, and Zephyr)
# Pull Request Checklist ## Reference Issue Please provide the reference to issue this PR is addressing (# followed by the issue number). If there is no associated issue, write...
# Pull Request Checklist `ServiceRetriever` class for interacting with Anserini REST API. Currently only supports retrieval from pre-built indexes and k=10 (REST API does not accept k as a parameter...
Hi there! Great work on these -- would love to try them out! I was curious if there is already a method/process for reranking given a list of pairs --...
# Reference URA-Project Issue 35 RankLLM: Merge LiT5 Models into RankLLM https://github.com/castorini/ura-projects/issues/35 The goal is to port LiT5-Distill and LiT5-Score over to RankLLM. ## Completed - Synced the LiT5 integration...
1. https://github.com/castorini/rank_llm/blob/d62d111784db4fea4312ca2a9941b91be80d057e/src/rank_llm/retrieve_and_rerank.py#L121 2. https://github.com/castorini/rank_llm/blob/d62d111784db4fea4312ca2a9941b91be80d057e/src/rank_llm/retrieve_and_rerank.py#L82C35-L82C48
Just thoughts... @ronakice @sahel-sh reactions welcome. Two methods: `rerank` and `rerank_batch`. Method signature: ``` def rerank(self, query: Query, candidates: Candidates, k: int = 10) -> Candidates: ``` Commentary: + I...
What is RankLLM? I can think of two obvious answers: **Approach 1. RankLLM is a fully-integrated layer on top of Anserini and Pyserini.** If this is the case, then we...
Hi, I wonder the time profiling of each LLMs to run across queries for re-ranking. I am running RankVicunna and RankZephyr on the zero-shot setting across BEIR datasets. For FiQA...
Hi, Where can I find the ranking fine-tuning script for both RankVicunna and RankZephyr ? Thanks in advance!
# Pull Request Checklist ## Reference Issue ref: https://github.com/castorini/ura-projects/issues/4 ## Checklist Items Before submitting your pull request, please review these items: - [Yes] Have you followed the [contributing guidelines](CONTRIBUTING.md)? -...
