haystack-core-integrations icon indicating copy to clipboard operation
haystack-core-integrations copied to clipboard

Support the FastEmbed GPU implementation

Open aymbot opened this issue 1 year ago • 0 comments

Is your feature request related to a problem? Please describe. Currently the Fastembed.... embedders are not utilizing the GPUs which makes it so that i.e. SPLADE takes a substantial amount of time for embeddings vs. its counterparts.

Describe the solution you'd like QDrant supports GPUs with another library, see here. Utilizing that library would allow us to leverage our GPUs. GPU-mode could be enabled with a flag or another method.

Describe alternatives you've considered Besides the Fastembed.. embedders, there are no out-of-the-box, nor integration, alternatives for sparse embeddings, meaning the only alternative would be to not use GPUs.

Additional context For now, none.

aymbot avatar Jul 16 '24 08:07 aymbot