candle

candle copied to clipboard

Hi, This is my Cargo.toml:

[package]
name = "myapp"
version = "0.1.0"
edition = "2021"

[dependencies]
candle-core = { git = "https://github.com/huggingface/candle.git", version = "0.5.0", features = ["cuda"] }

And this is my code:

use candle_core::{Device, Tensor};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let device = Device::new_cuda(0)?;

    let a = Tensor::randn(0f32, 1., (2, 3), &device)?;
    let b = Tensor::randn(0f32, 1., (3, 4), &device)?;

    let c = a.matmul(&b)?;
    println!("{c}");
    Ok(())
}

When compiling the code, an error occurred:

(py311) [wangjw@localhost myapp]$ cargo run
   Compiling cudarc v0.10.0
   Compiling candle-core v0.5.0 (https://github.com/huggingface/candle.git#b2e81675)
   Compiling myapp v0.1.0 (/home/wangjw/data/work/projects/practice/myapp)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 7.96s
     Running `target/debug/myapp`
target/debug/myapp: error while loading shared libraries: libnvrtc.so.12: cannot open shared object file: No such file or directory

The libnvrtc.so.12 is in "/home/wangjw/programs/miniconda3/envs/py311/lib", how to specify the path ?

You can probably use the LD_LIBRARY_PATH environment variable for this.

Yes, now it can be compiled. but I failed to run the example because of this error:

(py311) [wangjw@localhost myapp]$ cargo run
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/myapp`
Error: Cuda(Cublas(CublasError(CUBLAS_STATUS_NOT_INITIALIZED)))

Never seen this error, it seems to come from the cublas library that candle uses for matmuls on cuda, my guess is that there is an issue with the way cuda is installed on the box where you're trying to run this.