kernel-memory icon indicating copy to clipboard operation
kernel-memory copied to clipboard

Generate embeddings using batch requests

Open dluc opened this issue 1 year ago • 2 comments

Original issue: https://github.com/microsoft/semantic-kernel/issues/2168 from @Nurgo

Hi,

I've been working with SemanticKernel and I appreciate all the hard work you've put into this amazing project.

I'm currenly encountering a limitation with the ISemanticTextMemory.SaveInformationAsync method which only processes one text segment at a time, causing many calls to OpenAI's APIs when indexing large documents.

Would it be possible to modify this function to allow batch processing in a single call? Since IEmbeddingGeneration.GenerateEmbeddingsAsync already supports batching, it might be a feasible and impactful improvement.

Thanks for considering this suggestion.

dluc avatar Aug 21 '23 16:08 dluc