Build Faster Whisper in a docker container issues

[bakermanbrian]

trafficstars

I've been able to run regular Whisper just fine from a docker build, but haven't been able to get it to work for Faster Whisper

Currently using pytorch/pytorch:2.1.0-cuda11.8-cudnn8-runtime as the base image and getting the issue "libcudnn_cnn_infer.so.8 not found"

The nvidia image mentioned in the README was too large for my VM to handle

[funboarder13920]

You can try starting from the ctranslate2 docker image

[bakermanbrian]

Seems like the newest one of those images uses CUDA11.2, which I'm not sure will work for me here. I updated my docker file like so (I got the LD_LIBRARY_PATH by testing on my VM by running python3 -c 'import os; import nvidia.cublas.lib; import nvidia.cudnn.lib; print(os.path.dirname(nvidia.cublas.lib.__file__) + ":" + os.path.dirname(nvidia.cudnn.lib.__file__))' )

FROM pytorch/pytorch:2.1.0-cuda11.8-cudnn8-runtime

RUN pip install nvidia-cublas-cu11 nvidia-cudnn-cu11

RUN pip install -r requirements.txt

ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/conda/lib/python3.10/site-packages/nvidia/cublas/lib:/opt/conda/lib/python3.10/site-packages/nvidia/cudnn/lib

Got a bit further, but got this error:

Could not load library libcudnn_cnn_infer.so.8. Error: /opt/conda/lib/python3.10/site-packages/nvidia/cudnn/lib/libcudnn_cnn_infer.so.8: undefined symbol: _ZN11nvrtcHelper4loadEb, version libcudnn_ops_infer.so.8

Seems like it's close, just missing one thing on compatibility.

[lem21h]

This worked for me:

FROM nvidia/cuda:12.2.2-cudnn8-runtime-ubuntu22.04

ENV PYTHON_VERSION=3.10

RUN export DEBIAN_FRONTEND=noninteractive \
    && apt-get -qq update \
    && apt-get -qq install --no-install-recommends \
    python${PYTHON_VERSION} \
    python${PYTHON_VERSION}-venv \
    python3-pip \
    && rm -rf /var/lib/apt/lists/*

RUN ln -s -f /usr/bin/python${PYTHON_VERSION} /usr/bin/python3 && \
    ln -s -f /usr/bin/python${PYTHON_VERSION} /usr/bin/python && \
    ln -s -f /usr/bin/pip3 /usr/bin/pip

WORKDIR /app

COPY docker/requirement_whisper.txt ./requirements.txt
RUN pip install --no-cache-dir -r requirements.txt
...
ENTRYPOINT "/app/entrypoint.sh"

And then in entrypoint.sh I have:

export LD_LIBRARY_PATH=`python -c 'import os; import nvidia.cublas.lib; import nvidia.cudnn.lib; print(os.path.dirname(nvidia.cublas.lib.__file__) + ":" + os.path.dirname(nvidia.cudnn.lib.__file__))'`

python -u main.py

or just add this to docker compose that LD_LIBRARY_PATH

[bakermanbrian]

Unfortunately the nvidia/cuda:12.2.2-cudnn8-runtime-ubuntu22.04 docker image is too large for my machine to handle; still not sure why the original pytorch image doesn't accomplish what I need. Do I need to be setting the LD_LIBRARY_PATH (I'd assume that would be set in the base image)?

[lem21h]

setting LD_LIBRARY_PATH is needed due the fact that the libraries that are needed are installed via PIP (not by operating system). In other words if those libraries were installed by apt (as base is UBUNTU), you would not need to set that LD_LIBRARY_PATH

This don't require downloading those python packages and makes faster-whisper still work:

FROM nvidia/cuda:12.2.2-cudnn8-runtime-ubuntu22.04

ENV PYTHON_VERSION=3.10

RUN export DEBIAN_FRONTEND=noninteractive \
    && apt-get -qq update \
    && apt-get -qq install --no-install-recommends \
    python${PYTHON_VERSION} \
    python${PYTHON_VERSION}-venv \
    python3-pip \
    libcublas11 \
    && rm -rf /var/lib/apt/lists/*

[sayedmohamedscu]

this worked for me after trying multiple solutions

FROM pytorch/pytorch:1.13.1-cuda11.6-cudnn8-runtime

WORKDIR /app
COPY . .

RUN apt-get update && apt-get install -y libglib2.0-0 build-essential git && \
    apt-get clean && rm -rf /var/lib/apt/lists/*

ENV TORCH_CUDA_ARCH_LIST="6.0;6.1;7.0;7.5;8.0;8.6+PTX;8.9;9.0"

RUN pip install -U pip setuptools
RUN pip install python-multipart faster-whisper nvidia-cublas-cu11 nvidia-cudnn-cu11

ENTRYPOINT ["sh", "-c", "export LD_LIBRARY_PATH=`python -c 'import os; import nvidia.cublas.lib; import nvidia.cudnn.lib; print(os.path.dirname(nvidia.cublas.lib.__file__) + \":\" + os.path.dirname(nvidia.cudnn.lib.__file__))'` && python -u test.py"]

[BBC-Esq]

Ctranslate2 just release version 4.0 that now has CUDA 12+ support, just FYI. Not sure what faster-whisper will need to do to match but if there's anything I can do to help...

https://github.com/OpenNMT/CTranslate2/releases/tag/v4.0.0

[fedirz]

Here's a workaround, which might help for those trying to make it work in Docker.

https://github.com/SYSTRAN/faster-whisper/issues/516#issuecomment-2119010765

[iantanwx]

fwiw, nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04 fixed it for me, without having to set library paths. i was using it through whisperx.