Eric Buehler
Eric Buehler
@terhechte thanks for this. I merged #1252 - this should fix the issue, can you please try again after `git pull`?
@aminnasiri can you please use `git pull` for the latest version and recompile? I merged #1129 which gates our new NCCL support behind a feature flag for build compatibility -...
Hi @aminnasiri ! Can you please recompile with NVCC_CUDA_FLAGS="-fPIC" (#286)
@aminnasiri can you please paste the error/log?
@aminnasiri this has been fixed, can you please try it again after `git pull`?
@aminnasiri the current latest commit is [c9ac321](https://github.com/EricLBuehler/mistral.rs/commit/c9ac3213264be0dbbe010ad0035715a563b64bb8). If running `git pull` does not work to fetch these latest changes, can you delete & re-clone the repository?
@aminnasiri did you specify the NVCC_CUDA_FLAGS variable (i.e. `NVCC_CUDA_FLAGS="-fPIC" cargo build --release --features "cuda flash-attn"`)?
@maximizemaxwell you can look at: https://github.com/EricLBuehler/mistral.rs/blob/4608202c128da44b84157573dbc8ff1a1146f64c/mistralrs-core/src/layers.rs#L1965-L2036 This is written under the assumption that the temporal patch size==`kernel_sizes[0]`==2, which is true for Qwen 2.5 VL.
\image https://upload.wikimedia.org/wikipedia/commons/thumb/3/3a/Rosa_Precious_platinum.jpg/220px-Rosa_Precious_platinum.jpg What is this image?
Unfortunately not, Candle cannot load the MXFP4 checkpoints at the moment. We're working on a fix.