fastembed-gpu and 'CUDAExecutionProvider' not working as expected:

[Gautam-Rajeev]

Hi! Just raising this as an issue just in case it gets lost.

Have tried it fastembed-gpu in collab and it doesn't seem to be using GPU (only the CPU mem usage is increasing, GPU is remaining 0)

Have attached the collab link here - https://colab.research.google.com/drive/1I4Z547eE0utHmp2pxIgaLx0PXQhNFWdj?usp=sharing

Thanks a lot!

[joein]

Hi @GautamR-Samagra , yes, it requires some tweaking because colab has cuda 12.x. You would need to install onnxruntime-gpu separately with a url, and then install fastembed, you can find instructions in readme

[amkazan]

did as in instruction. still does not use GPU

[joein]

Hi @amkazan, I'll provide a notebook

[joein]

@GautamR-Samagra @amkazan It seems like the recent update of onnxruntime-gpu has broken the link, I think it will be fixed on their side soon. I had to fix onnxruntime-gpu to 1.17.1, here is a working notebook

[Gautam-Rajeev]

@GautamR-Samagra @amkazan It seems like the recent update of onnxruntime-gpu has broken the link, I think it will be fixed on their side soon. I had to fix onnxruntime-gpu to 1.17.1, here is a working notebook

Thank you so much, works on GPU crazy fast!

[joein]

https://github.com/microsoft/onnxruntime/issues/20745