mistral.rs icon indicating copy to clipboard operation
mistral.rs copied to clipboard

Published 20 hours ago • verified publisher icon EricLBuehler

Error: DriverError(CUDA_ERROR_INVALID_PTX, "a PTX JIT compilation failed") when loading utanh_bf16

Open nikolaydubina opened this issue 4 months ago • 5 comments

Describe the bug

LLAMA 3.2 11B Vision cannot start after loading model

Error: DriverError(CUDA_ERROR_INVALID_PTX, "a PTX JIT compilation failed") when loading utanh_bf16
image

my system

DRIVER_VERSION=550.90.07

Latest commit or version

ghcr.io/ericlbuehler/mistral.rs:cuda-90-sha-1caf83a@sha256:095518a16d1f0a9fa2e212463736ccb540eeb0f88f21c10a2123ab8cf481b83e

References

  • 550.90.07 driver supports 12.4 cuda, which is same as in Docker image 12.4.1 https://docs.nvidia.com/datacenter/tesla/tesla-release-notes-550-90-07/index.html image

nikolaydubina avatar Oct 14 '24 06:10 nikolaydubina