torchchat icon indicating copy to clipboard operation
torchchat copied to clipboard

Implement the AO API in torchchat quantization handlers and unify logic.

Open mikekgfb opened this issue 4 months ago • 1 comments

Implement the AO API in torchchat quantization handlers and unify logic.

1 - implement .quantize() for TC quantization handlers and support args to make consistent with AO 2 - remove special handling for various combinations of parameters and use validate_args before calling with **q_kwargs 3 - remove check probing whether we successfully loaded a8wx and install an error-reporting handler if loading failed which will be called as quant handler and issue an error 4 - unify both tc and ao quantization handler dicts with shared calling logic 5 - provide informational message when quantizer option not supported (via introspection)

mikekgfb avatar Oct 10 '24 20:10 mikekgfb