text-embeddings-inference icon indicating copy to clipboard operation
text-embeddings-inference copied to clipboard

Could auto-truncate also deal with character limits?

Open qherreros opened this issue 6 months ago • 0 comments

Feature request

Today, truncation only deals with the number of tokens. It doesn't manage the hard character limit (250*n_tokens). This limit seems high but it does fail on some datasets (NQ for example, probably because of URLs contained in documents).

Motivation

It would avoid having truncation logic outside the endpoint.

Your contribution

I could try to open a PR but I'm not fluent in rust sadly.

qherreros avatar Jul 29 '24 08:07 qherreros