disbullief comments

Repositories
Issues
Comments

Results 3 comments of


                                            disbullief

[REQUEST] Model serving via deepspeed's inference module

Any update or workaround?

Could not load library libcudnn_ops_infer.so.8

For everyone that has this issue, what fixed it for me was to include the path to torch too in `LD_LIBRARY_PATH`. The docker image I run is `pytorch/pytorch:2.0.1-cuda11.7-cudnn8-runtime` The line...

Could not load library libcudnn_ops_infer.so.8

@otonoton I think that your best bet would be to use [Whisper C++](https://github.com/ggerganov/whisper.cpp)