candle
candle copied to clipboard
huggingface
Minimalist ML framework for Rust
## Summary Implements `BertForTokenClassification` for pure-Rust Named Entity Recognition (NER) in Candle, following the existing DeBERTa V2 pattern. This enables Rust applications to perform token classification tasks (NER, POS tagging,...
This PR adds CUDA graph support. Currently it's under development, I'll add more details here when the structure seems final, until then this PR is just for tracking progress. Related...
The Candle ecosystem appears to require developers to define their own plumbing for inter-process communication and process-level concerns such as logging and error handling. 1. Utilizing standardized abstractions via enums...
## Description: On the Metal backend, large transformer/VLM workloads (e.g. Dots‑style OCR with a heavy vision tower and Qwen2‑style text tower) can cause the process RSS on macOS to grow...
Set math mode via old api if macos version < 15 or ios < 18. Fixes #3185
# Description ## Summary Okay, so looks like I stumbled upon an obscure bug here. I tried reducing the bug to as small as an MRP as possible, but it...
The commit 06387ae55 adds a call to compile_options.setMathMode() that is only supported from MacOS 15.0 and up. I am using Candle for an application that requires support for older MacOS...
## Summary PR #3143 introduces `ConcatKvCache` which provides **2-5x GPU speedup** for autoregressive generation with no breaking changes to the API. However, it hasn't been reviewed yet. **This issue aims...
