text-embeddings-inference icon indicating copy to clipboard operation
text-embeddings-inference copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

Support for GPU and CPU in one Docker image

Open saileshd1402 opened this issue 8 months ago • 2 comments

Feature request

I would like to request to have a single docker image for both CPU and GPU cases. This can be done using a combination of Dockerfile and Dockerfile-cuda-all. An entrypoint.sh can choose between CPU and GPU binaries based on availability of CUDA drivers or based on "CUDA_VISIBLE_DEVICES".

Please let me know your thoughts on this

Motivation

This would help in not always configuring the images to use based on the resources. The image size would be slightly bigger but I think that is a decent trade-off

Your contribution

I would like to help contribute this with a PR, if it's an acceptable feature.

saileshd1402 avatar Feb 14 '25 11:02 saileshd1402