triton-inference-server
Triton Server 24.05 can't initialize CUDA drivers if host system has installed Nvidia driver 555.85
Description
I was using Triton Server nvcr.io/nvidia/tritonserver:24.04-py3 on my local machine with Windows 10 via docker container. Ie installed latest Nvidia Driver 555.85, and docker container stopped to run, returning error: ERROR: The NVIDIA Driver is present, but CUDA failed to initialize. GPU functionality will not be available. [[ Named symbol not found (error 500) ]]
I downloaded container with version 24.05, but it didn't help - server doesn't start, returns same error instead.
Triton Information tritonserver:24.04-py3, tritonserver:24.05-py3,
Are you using the Triton container or did you build it yourself? Container
To Reproduce Windows 10, install Nvidia driver 555.85, run Triton Server tritonserver:24.05-py3 with model.
Describe the models (framework, inputs, outputs), ideally include the model configuration file (if using an ensemble include the model configuration file for that as well). Model: efficientnetv2 converted to onnx from tflite.
backend: "onnxruntime"
max_batch_size: 32
input [
{
name: "serving_default_input_1:0",
data_type: TYPE_FP32,
dims: [ 512, 512, 3 ]
}
]
output [
{
name: "StatefulPartitionedCall:0",
data_type: TYPE_FP32,
dims: [ 2 ]
}
]
dynamic_batching { }
response_cache { enable: true }
Expected behavior Triton Server has to run my model.
The NGC containers are only supported on Linux. For Windows, please refer to the zip file that is delivered as a part of Github release: https://github.com/triton-inference-server/server/releases/tag/v2.44.0 (24.03 is the latest release of Windows support).
The NGC containers are only supported on Linux. For Windows, please refer to the
zipfile that is delivered as a part of Github release: https://github.com/triton-inference-server/server/releases/tag/v2.44.0 (24.03 is the latest release of Windows support).
Triton Server properly worked with previous version of driver. I downgraded driver already and Triton started to work again. I'm using 24.05 NGC Triton container.
Do you want to say that 24.03 will work with driver 555.85, and later Triton Server versions won't be working with latest Windows Nvidia drivers anymore?
我也遇到了相同的问题，你是在wsl2里运行的吗？ 我之前triton服务可以正常推理，但是更新了驱动，现在是560的显卡驱动，用的23.09的镜像不可以，并且尝试了多种镜像，包括24.5 24.9 都不可以，请问你是怎么降级的显卡驱动啊，我在英伟达官网上看3090ti的最低的能下载的驱动都是551.86
我也遇到了相同的问题，你是在wsl2里运行的吗？ 我之前triton服务可以正常推理，但是更新了驱动，现在是560的显卡驱动，用的23.09的镜像不可以，并且尝试了多种镜像，包括24.5 24.9 都不可以，请问你是怎么降级的显卡驱动啊，我在英伟达官网上看3090ti的最低的能下载的驱动都是551.86
Yes, I used WSL2. As I understand, latest versions of Triton can be run on Windows using .zip version of release. However I didn't try them, so I can't tell is it difficult to setup that version or not.
My Triton worked propely on 552.22 version, so maybe you can try that version too. I'm not sure, how to downgrade Nvidia dirver, I don't remember how I did it. Probaly you can try to install older version? If it won't work, I think there are some guides online for that.
