Benjamin Fineran

icon indicating a website link https://www.linkedin.com/in/benjaminfineran

icon company Scale AI icon location ML @scaleapi | formerly @neuralmagic @google, @facebook, @upenn

Results 11 issues of Benjamin Fineran

HFQuantizer implementation for compressed-tensors library

3 comment

This PR adds an `HFQuantizer` for the [compressed-tensors](https://github.com/neuralmagic/compressed-tensors) library. Supported quantization features include: * FP8, INT4, INT8 (for Q/DQ arbitrary precision is allowed for INT) * Activation quantization (static) *...