mistral.rs

mistral.rs copied to clipboard

MODEL_ID not "MODEL_ID"

1

simonw

#832 - Unable to build on Windows with VS and CUDA 12.6

5

## Minimum reproducible example cargo build --release --features cuda ## Error Build error candle 0.7.2 error trace: error: failed to run custom build command for `candle-kernels v0.7.2 (https://github.com/EricLBuehler/candle.git?rev=60eb251#60eb251f)` note: To...

misureaudio

Error: DriverError(CUDA_ERROR_INVALID_PTX, "a PTX JIT compilation failed") when loading utanh_bf16

5

## Describe the bug LLAMA 3.2 11B Vision cannot start after loading model ``` Error: DriverError(CUDA_ERROR_INVALID_PTX, "a PTX JIT compilation failed") when loading utanh_bf16 ``` my system ``` DRIVER_VERSION=550.90.07 ```...

nikolaydubina

Add Phi 3.5 MoE, Mixtral support for UQFF

1

EricLBuehler

fix Half-Quadratic Quantization and Dequantization on CPU

3

This confirms that `test_bitpack` is running solely on non-CPU hardware. To address this, we could implement a fix by ensuring contiguous data slices.

haricot

add marlin as cargo feature

1

This ensures broader accessibility to Mistral.rs across different hardware configurations by enabling its use even on systems with a CUDA compute capability less than 8.0.

haricot