Nicolas PASCAL

add marlin as cargo feature

1

This ensures broader accessibility to Mistral.rs across different hardware configurations by enabling its use even on systems with a CUDA compute capability less than 8.0.

The previous implementation of `to_dtype` used a large `match` statement. This commit replaces the `match` with calls to specialized `as_` methods, which contain highly optimized conversion logic. This significantly improves...

paged-attn:cuda fallback bf16 for compute_cap < 8.0

1

tested with PR https://github.com/EricLBuehler/candle/pull/57 and works with: ``` nvidia-smi --query-gpu="compute_cap" --format=csv compute_cap 6.1 ``` ``` cargo run -F "cuda cudnn" -r -- --throughput -i plain -m meta-llama/Llama-3.2-1B-Instruct --dtype bf16 cargo...

CUDA: Add (full BF16) and FP8 fallback support for compute >=530

5

tested and works with: ``` nvidia-smi --query-gpu="compute_cap" --format=csv compute_cap 6.1 ``` ``` cargo run -F "cuda,cudnn" --example llama --release -- --model-id meta-llama/Llama-3.2-1B-Instruct --temperature 0.1 --which v32-1b-instruct --seed 42 --dtype bf16...