Niklas

Hey are you reffering to the `crossencoder_openai.ipynb` notebook? You can still use the OpenAI search endpoint using the workaround in [this script](https://github.com/Muennighoff/sgpt/blob/main/crossencoder/beir/openai_search_endpoint_functionality.py) using their completion endpoint.

Yeah you probably need to remove the `print(openai.Search.create(model="davinci", query=query, documents=docs))` part & the BM25 is missing I think. It may be cheaper for you to just use the Cross-Encoder method...

> Hi @Muennighoff, I would like to use your cross encoder with different GPT models. I have noticed that this [script](https://github.com/Muennighoff/sgpt#cross-encoder) is different from the code in the [notebook](https://github.com/Muennighoff/sgpt/blob/main/crossencoder/beir/crossencoder_beir_sgpt.ipynb). Could...

This is cool! But I think we should integrate it with the existing MTEB interface or does that not make any sense? cc'ing @dipam7 @KennethEnevoldsen from the issue

Here's a great example of a successful integration: https://github.com/embeddings-benchmark/mteb/pull/408 Tagging @orionw here in case he has ideas 😊

Also cc'ing @hongjin-su here who also knows mteb quite well & may be interested in adding / helping add this

> The performance for prompt retrieval is measured by LLM results in downstream tasks. Back then, the paper used GPT-J. Should we switch to a more up-to-date model, e.g., Llama3-8B...

Really great idea! What do you think about making this a new repository? It could become something like a library of Embedding Models (including APIs). Afaict there is no such...

> Just to start of with, I completely agree the the notion of keeping libraries lightweight and that this allows for better scaling. > > The goal of the proposed...

That'd be great indeed cc @thakur-nandan