rank_llm icon indicating copy to clipboard operation
rank_llm copied to clipboard

Easy way to run models on a list of examples?

Open orionw opened this issue 9 months ago • 2 comments

Hi there! Great work on these -- would love to try them out!

I was curious if there is already a method/process for reranking given a list of pairs -- it seems like I would need to build out a Request object with candidates, but I'm not 100% certain.

Is there any minimal example scripts? Or perhaps one that loads from a JSON file that I can emulate?

Currently the README only has the one example that is fairly non-intuitive for new data that isn't an existing TREC dataset: I have to dig through several layers to find out what the inputs and outputs of these functions are and it still isn't quite clear how I could load my own data into it.

Any help would be greatly appreciated and I'm sure would be useful to the broader community as well!

orionw avatar May 10 '24 21:05 orionw