Is there any doc about config file of QuantizationSimModel?

[lcybuzz]

Hi, I found a introduction in https://quic.github.io/aimet-pages/releases/1.16.2/user_guide/quantization_sim.html#ug-quantsim and an example in default_config. However, I still don't understand what's the meaning of each section and how to write my own configurations. Where can I find more detailed explanation? Thanks for any help.

[quic-ssiddego]

Hi @lcybuzz Thank you for your query. Here's the schema definition for the config file :https://github.com/quic/aimet/blob/develop/TrainingExtensions/common/src/python/aimet_common/quantsim_config/quantsim_config_schema.py . default_config shows an example config. We are working on adding a detailed specification for same. For now, if you can share the specific list of the configs you would like to add, we could help you with it.

[lcybuzz]

Hi @quic-ssiddego. Thank you for your reply! I'm trying to deploy TF1 model on Hexagon DSP with SNPE. I compare model.encodings got by running QuantizationSimModel.export() and quantized DLC, and find that some activataions have no encoding info in the file. The figure below shows an example.

Currently, I set config_file=None and I want to add the following configurations to make QuantSimModel have the same quantization scheme with SNPE:

1. Quantize conv separately regardless of its groups such as conv-relu, conv-add and so on.
2. Bias parameters are quantized.
3. activations of add are quantized.
4. parameters (weights and bias) of conv2d_transpose are quantized.

Could you please help me on this? Thanks a lot.

[quic-ssiddego]

Hi @lcybuzz When the config file is set to none, we apply default_config rules. For this case, you could make a copy of the default_config.json and edit it to reflect your specific rules. Reference config file : https://github.com/quic/aimet/blob/develop/TrainingExtensions/common/src/python/aimet_common/quantsim_config/default_config.json ,

• If you want to skip operator fusing, just edit the supergroup rule to exclude conv-relu :

• Remove this entry "supergroups": [

  { "op_list": ["Conv", "Relu"] }, ... ],

• To control the bias quantization use, set this to true if you need bias quantization

"params": { "bias": { "is_quantized": "False" } },

• For add to be quantized, use super groups rule or individual op rule
  check this example :https://github.com/quic/aimet/blob/develop/TrainingExtensions/torch/test/python/test_quantsim_config.py#L282

• use above for conv2d_transpose as well. (another example here : https://github.com/quic/aimet/blob/develop/TrainingExtensions/torch/test/python/test_quantsim_config.py#L164)

Please let us know if you have further queries.

[lcybuzz]

Hi @quic-ssiddego. Under your instructions, I have successfully solved the first three questions. However, I still have some problems for " adding parameter quantization nodes for conv2d_transpose". I found op types with weights here defining OP_WEIGHT_TYPES = ['Conv2D', 'MatMul', 'DepthwiseConv2dNative', 'Conv2DTranspose']. It's used for checking if a given op has weights. Actually for conv2d_transpose op, the type name is Conv2DBackpropInput instead of Conv2DTranspose (TF v1.15.0). As a result, AIMET doesn't treat it as an op with parameters and fails to insert param quantizers. I think this may be the reason for this issue. Is there any method solving this? Thanks again for your help.

[quic-ssiddego]

@icybuzz Is this a TF 2.x op? AIMET does not support TF 2.x at the moment. We are in the process of migration. A quick workaround would be to use compatible version tf.compat.v1.raw_ops.Conv2DBackpropInput and/ update above OP_WEIGHT_TYPES list and give it a try. This would also require an entry to the op type pattern list we support, here: https://github.com/quic/aimet/blob/develop/TrainingExtensions/tensorflow/src/python/aimet_tensorflow/common/sub_graph_matcher_op_templates.py along with a valid unit test case (example : https://github.com/quic/aimet/blob/develop/TrainingExtensions/tensorflow/test/python/test_tf_connected_graph_with_subgraph_matcher.py#L69) . If this works for you, please go ahead and create a PR to contribute this back to AIMET. Appreciate your feedback and contribution. Thank you.

[lcybuzz]

Hi @quic-ssiddego. I use TF 1.15.0, so I think it should be a TF 1.x op. Besides, I have tested several APIs for transpose convolution, including tf.nn.conv2d_transpose, tf.layers.conv2d_transpose, tf.layers.Conv2DTranspose and tf.keras.layers.Conv2DTranspose. All of them have the type Conv2DBackpropInput. I will try under your advise and give feedback if it works. Thanks~

[quic-ssiddego]

@icybuzz If it's TF 1.x, then you would need to update the two lists and be able to insert the param quantizers. Please give it a try and let us know.

[lcybuzz]

Hi @quic-ssiddego. I have tried and successfully inserted quantizers for transpose conv. I have made the following modifications:

1. Replace Conv2DTranspose with Conv2DBackpropInput in OP_WEIGHT_TYPES and OP_WEIGHT_INDICES here
2. Replace op_type: 'Conv2DTranspose' with op_type: 'Conv2DBackpropInput' of Conv2DTranspose and Conv2DTranspose_with_bias here
3. Replace Conv2DTranspose with Conv2DBackpropInput here

Now weights and bias encodings of transpose conv have appeared in the exported encoding file. But I haven't tested the performance for QAT with the new configuration. Besides, in my case, I was suprised to find that sometimes performances of models with all desired quantizers are slightly worse than those with some quantizers missing.

Thanks a lot for the help.

[quic-ssiddego]

@Icybuzz Apologeies for the delayed response. Thank you for the update. It would be great if you could add a corresponding unit test and create a pull request for merging this back to AIMET. What you observe seems to be a valid observation. With reduced bitwidths, depending on the model, there could be an impact on the performance.

[lcybuzz]

@quic-ssiddego I'm very glad to contribute to this project. However, I have never submit codes to a public repo before. I think I need to get familar with the regulations first. Then I will come back and create a PR if this issue is still not solved. Thank you very much~

[quic-ssiddego]

@Icybuzz Please let me know if you have any questions. Will be happy to assist you with this contribution. Thank you.

[quic-mangal]

Closing this issue due to inactivity. Please re-open it/ create a new issue if you need further help.