AMDMIGraphX icon indicating copy to clipboard operation
AMDMIGraphX copied to clipboard

Introduce `--int4-weights` option in `migraphx-driver`. This would require changes in MIGraphX's naive quantizer to set range between `[0, 15]` During quantization it should also insert "pack" and "unpack" instructions.

Open lakhinderwalia opened this issue 1 year ago • 1 comments

lakhinderwalia avatar Aug 02 '24 23:08 lakhinderwalia

Since its quantizing the weights we dont need to use our quantizer. Instead we would just take the range of weights and compute scale so it fits in the range of [0, 15]. We also need to insert Q/DQ pairs as well.

pfultz2 avatar Aug 05 '24 14:08 pfultz2