lmdeploy
lmdeploy copied to clipboard
InternLM
Are there any plans to support CUDA 11.7?
Motivation
Hi, currently there seems to only be support for CUDA 11.8 and 12. Currently our system only supports CUDA 11.7 which creates issues with the torch install. Does that mean that we currently cannot use lmdeploy?
Related resources
No response
Additional context
No response
Already did. The lmdeploy+cu118 package can be installed in the cuda 11.7 platform
That is true, however, since it is compiled on Cuda 11.8, the torch version that it installs must also be compiled on Cuda 11.8 and the default installation is torch 2.2.2+cu118. However, we have to use a torch installation that is compiled with Cuda 11.7 to be able to run the models since our nvcc version is Cuda 11.7.
It might be too specific of an error as the customizability of our hardware is limited, but perhaps there are already existing workarounds? What do you think? Thank you so much for your help on this!
If your nvidia driver only support cuda 11.7, you may install this way:
export LMDEPLOY_VERSION=0.4.1
export PYTHON_VERSION=38
pip install https://github.com/InternLM/lmdeploy/releases/download/v${LMDEPLOY_VERSION}/lmdeploy-${LMDEPLOY_VERSION}+cu118-cp${PYTHON_VERSION}-cp${PYTHON_VERSION}-manylinux2014_x86_64.whl --extra-index-url https://download.pytorch.org/whl/cu117
# there exists torch-2.0.1+cu117 whl package which meets lmdeploy requirement.
After installation, you should check the version of 'nvidia-nccl-cu11', 'nvidia-cuda-runtime-cu11', 'nvidia-cublas-cu11', 'nvidia-curand-cu11' python packages and make sure the version is lower than your nvidia driver supports.
