aimet
aimet copied to clipboard
Does aimet and qnn htp backend support weight-only quantization?
Does aimet and qnn htp backend support weight-only quantization? For example, the activation is fp16, and weight is 4bit/8bit quantized.
Or does qnn htp backend support fp16 matmul? If so, it may be feasible to manually dequantize the weight, and then perform fp16 matmul. The qnn operation documentation says that htp backend support fp16 fullyconnected, but I failed to run it on devices.