text-embeddings-inference icon indicating copy to clipboard operation
text-embeddings-inference copied to clipboard

Support for config_sentence_transformers.json

Open sam-ulrich1 opened this issue 1 year ago • 0 comments

Feature request

Add cli option to auto-format input text with config_sentence_transformers.json prompt settings (if provided) before toknizing.

Motivation

A lot of models now expect a prompt prefix so enabling the server-side handle of this allows clients to become model "model-agnostic". We have trouble changing between models since we must support the custom prompt for each specific model on the client side. Server side via config would remove this entriely.

Your contribution

Happy to do the PR myself just want to make sure this would be a welcome contribution

sam-ulrich1 avatar Apr 18 '24 15:04 sam-ulrich1