voyageai-python icon indicating copy to clipboard operation
voyageai-python copied to clipboard

[Reranker] Feature request: send in document id and possibly other metadata

Open fdurant opened this issue 1 year ago • 0 comments

I'm trying out the brand new Voyage reranker using the voyageai==0.2.3 Python client.

When calling

reranking = vo.rerank(query, documents, model="rerank-lite-1", top_k=3)

it would be super handy to pass in not only the document text, but also the original document ID and possibly other metadata. This way, when receiving the reranked results, there is no need to figure out afterwards which winning top-k text has which id. This facilitates downstream processing.

Practically speaking, the proposal would be to extend the typing of documents from List[str] to List[Dict[str, Any]] where the keys would be text (mandatory) and optionally doc_id (or similar), plus other metadata keys.

The result could then be of the same type as now, but with an additional key doc_id, plus the other metadata keys.

fdurant avatar Jun 17 '24 07:06 fdurant