candle
candle copied to clipboard
huggingface
Minimalist ML framework for Rust
Adds a new WASM example for running quantized Qwen3-0.6B models in the browser with WebAssembly SIMD optimizations. ### Features - **SIMD-optimized inference**: Leverages WASM SIMD128 instructions for accelerated matrix operations...
## Summary This PR adds comprehensive support for SmolLM3-3B with both full precision (safetensors) and quantized (GGUF) implementations, unified under a single example interface. ## What's New ### Model Implementation...
### Disclaimer This PR is still a **work in progress** and needs some polishing. The main goal of opening it early is to gather feedback on whether this approach is...
Hi candle devs! I'm using candle to inference a small model (Qwen-0.6B-4b) using CPU. Profiling the inference show that there's significant overhead in rayon processing, primarily because too fine-grained multi-thread...
Currently, the documentation is written in English and built with mdBook. It would be very helpful if the docs were structured in a way that supports multiple languages (for example,...
Candle version: main Platform: Mac Studio Max M1 Mode: Qwen 3-1.7b, (download by huggingface-cli) Execute cmd: git clone https://github.com/huggingface/candle.git cd candle-examples cargo run --release --example qwen -- \ --prompt "What...
Trying to run frame interpolation model with `candle_onnx`, and getting error `Unsupported resize mode: linear`. Example onnx node: ``` - /core_model/Resize: Resize - coordinate_transformation_mode: s="half_pixel", i=0 - cubic_coeff_a: f=-0.75, i=0...
Make get() use squeeze() instead of reshape(), just like get_on_dim() does. It should avoid a copy when the tensor is not contiguous. Also use the same name for the index...
I am working on serializing a neural net and I noticed that [`LayerNorm`](https://github.com/huggingface/candle/blob/461e8c1685e003bdddfd1e7d1aa5092786ca9df5/candle-nn/src/layer_norm.rs#L64) does not expose the `remove_mean` and `eps` parameters, so serializing this struct isn't possible. Is hiding these...
