candle icon indicating copy to clipboard operation
candle copied to clipboard

VarMap Overhaul: Mutex/RwLock Backends, Unified Interface, and Concurrency Support

Open artem1984A opened this issue 4 months ago • 1 comments

This PR introduces a unified and extensible interface and serialization format.

Key updates:

Adds ConcurrentVarMap (RwLock-based) for high-performance, concurrent inference.

Refactors VarMap to use a generic storage backend, preserving the original Mutex-based behavior for training.

Ensures full compatibility between old and new APIs, including save/load and variable initialization.

Comprehensive tests for compatibility, integration, and concurrent stress scenarios.

These changes make it easier to scale Candle models for both training and production inference, while keeping the API simple and backward-compatible.

artem1984A avatar Jul 19 '25 20:07 artem1984A