t2v-transformers-models icon indicating copy to clipboard operation
t2v-transformers-models copied to clipboard

Batching along different texts, producing one vector per text

Open JubilantJerry opened this issue 1 year ago • 0 comments

Does the logic here implement batch processing of many different independent texts (i.e. independent entries in the Weaviate database)?

I see batching in the sense of splitting a text into sentences, computing vectors for each sentence, and averaging the vectors. I imagine this is done when a single entry contains multiple concepts. But if I understand the code correctly, there is no way to get multiple independent vectors from several independent texts.

JubilantJerry avatar Apr 28 '23 06:04 JubilantJerry