[Feature Request]: Reranking/relevance labeling task support

[nihit]

Is your feature request related to a problem? Please describe.

Search usecases can broadly be classified into two buckets:

1. Lexical/Keyword based search - traditional IR systems, more or less what Lucene/Solr/ElasticSearch offer.
2. Semantic search - embedding the doc and query in some latent space, and using a vector DB for retrieving nearest neighbors for a query.

A hybrid approach is optimal for many usecases however - (i) stage 1 - lexical similarity based retrieval & L1 ranking (ii) LLM/semantic similarity based reranking (e.g. https://txt.cohere.com/rerank/).

Describe the solution you'd like Support for Reranking task in Autolabel. Given a Query, List of Documents, and guidelines to judge relevance, ask the LLM to rank the list of documents.

[aimetrics]

It would be more cool if autolabel supports prompting the LLM to rank the list of images!