mistral.rs Installation Error

happened when i entered: cargo run --release --features cuda -- -i --token-source none --isq Q4K plain -m microsoft/Phi-3-mini-4k-instruct -a phi3

error: failed to run custom build command for cudarc v0.11.4

Caused by: process didn't exit successfully: /root/mistral.rs/target/release/build/cudarc-7276cb883fd7b801/build-script-build (exit status: 101) --- stdout cargo:rerun-if-changed=build.rs cargo:rerun-if-env-changed=CUDA_ROOT cargo:rerun-if-env-changed=CUDA_PATH cargo:rerun-if-env-changed=CUDA_TOOLKIT_ROOT_DIR

--- stderr thread 'main' panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cudarc-0.11.4/build.rs:73:14: Unsupported cuda toolkit version: 11.5. Please raise a github issue. note: run with RUST_BACKTRACE=1 environment variable to display a backtrace warning: build failed, waiting for other jobs to finish... root@Glarbsttentenford:~/mistral.rs#

Jun 05 '24 16:06 TimDouglas2

Hi @TimDouglas2! I have opened an issue with cudarc and will let you know when it is resolved.

Jun 05 '24 22:06 EricLBuehler

Hello @TimDouglas2! I just merged #424 which uses a new version of our CUDA backend driver which should support version 11.5.

Can you please run

git pull
cargo update

And then try to run the model again?

Jun 11 '24 17:06 EricLBuehler

@TimDouglas2 tagging to let you know I am closing this.

Jul 19 '24 01:07 EricLBuehler

Hello @TimDouglas2! I just merged #424 which uses a new version of our CUDA backend driver which should support version 11.5.

Can you please run
git pull
cargo update
And then try to run the model again?

These commands seem to have fixed my "undefined reference to 'hgemm_' build error

Jul 29 '24 06:07 oldgithubman