Example of how exactly Quantization aware training could be applied

[aj74aguk]

Hi,

I'm having trouble of using distiller to quantize my own model. Could you make an example of how exactly Quantization Aware Training could be applied to our own model? The .yaml files from the quantization example is actually not helping since that's just a file without any execution properties.

Thanks in advanced!

[ghost]

Hi, I'm troubled by the same problem. Could you tell me how you work out this issue? thanks

[nzmora]

Hi,

Quantization-Aware Training (QAT) in Distiller is described in several places:

• Overview
• YAML syntax details
• Preparing a model for quantization
• Algorithm
• Design
• Example QAT YAML files

The example classifier compression sample application uses the YAML files as described in the documentation above. You need to learn how the quantization process is embedded in the training-loop and how the quantizer is configured from the YAML. This is explained in the links above, which contain a lot of information. We currently do not have an example of QAT using the direct API (i.e. without the YAML).

Cheers, Neta

[LucaUrbinati44]

Hi,

is there the possibility to effectively produce a quantized model from a QAT model? I read this "converting from a quantization-aware training model to a post-training quantization model is not yet supported", but I don't get which is the sense of using QAT if you can't export the final model. Can you explain me how to do it?

Thank you