deepset-ai
Allow passing provider option to FastEmbed to allow embedding on the GPU
Is your feature request related to a problem? Please describe. I can't run the embedder on GPU so it is ridiculously slow
Describe the solution you'd like Right now the DocumentEmbedder and TextEmbedder don't have the option to pass the provider for a model to the backend that creates the model: https://github.com/deepset-ai/haystack-core-integrations/blob/9db9ca1ffb6f52b9af421927aac9376dcbafabd4/integrations/fastembed/src/haystack_integrations/components/embedders/fastembed/embedding_backend/fastembed_backend.py#L48-L50
This is the standard way to enable GPU support according to the FastEmbed docs:
embedding_model = TextEmbedding(
model_name="BAAI/bge-small-en-v1.5", providers=["CUDAExecutionProvider"]
)
Seems like it would be a ~6 line change
@alexkreidler Thank you for the feature suggestion. Based on the fastembed documentation, my understanding is that this feature would require pip install fastembed-gpu. fastemebd-gpu cannot be installed in the same environment as fastembed.
I wonder whether a separate integration, fastembed-gpu-haystack, makes sense here because of the conflicting dependencies.
Would you use such a feature with a single GPU or multi-GPU? I am asking because for multi-GPU, we would want to expose more/other parameters based on this example from fastembed documentation.
