Add working example for applying CLE and BC for a PyTorch resnet18 model

[quic-akhobare]

• Create a new examples directory at the top-level
• Use the API doc examples to apply CLE and BC quantization techniques for a torchvision resnet18 model

If you are interested in working on this issue - please indicate via a comment on this issue. It should be possible for us to pair you up with an existing contributor to help you get started.

From a complexity perspective, this ticket is at an easy level.

[YouYueHuang]

Hi @quic-akhobare,

I am interested in working on this issue. I have experience in model compression using Pytorch and TensorflowLite. The techniques I used includes channel pruning and SVD.

[quic-akhobare]

Hi @YouYueHuang.. Awesome to know that you are interested in working on this. I assigned the ticket to you. If you have questions, please discuss via comments on this ticket.

Hi @quic-mangal - Please help @YouYueHuang if needed.

[quic-mangal]

HI @YouYueHuang, let me know if you have any questions.

[YouYueHuang]

@quic-mangal , @quic-akhobare. Thanks for the chance to help me get started. May I know what would be the expected deliverable?

[quic-akhobare]

@quic-mangal , @quic-akhobare. Thanks for the chance to help me get started. May I know what would be the expected deliverable?

Hi @YouYueHuang. Ok, so the expected deliverable is a working code example. Idea is that we publish this example for AIMET users to easily see a working code snippet for a specific use case. Currently there is no equivalent for this in the AIMET codebase though there are toy code snippets that you can see in the API docs.

For this particular ticket, you can use an existing torchvision model, e.g. resnet18 or resnet50. Then you write some code to evaluate the model using imagenet dataset. Then invoke the cross-layer equalization APIs to optimize this model for quantization. We can do quantized eval for the model before and after cross-layer equalization to see the improvement in accuracy. The last part @quic-mangal can guide you if needed (though API docs might be sufficient).

What do you think? Does this sound feasible to you?

[ibei]

I am interested in this issue. I am working a Model-Quant project based on Pytorch and tensorflow. the techniques include CLE, BC and QAT.

[YouYueHuang]

Hi @YouYueHuang. Ok, so the expected deliverable is a working code example. Idea is that we publish this example for AIMET users to easily see a working code snippet for a specific use case. Currently there is no equivalent for this in the AIMET codebase though there are toy code snippets that you can see in the API docs.

For this particular ticket, you can use an existing torchvision model, e.g. resnet18 or resnet50. Then you write some code to evaluate the model using imagenet dataset. Then invoke the cross-layer equalization APIs to optimize this model for quantization. We can do quantized eval for the model before and after cross-layer equalization to see the improvement in accuracy. The last part @quic-mangal can guide you if needed (though API docs might be sufficient).

What do you think? Does this sound feasible to you?

Hi @quic-akhobare , this is feasible to me. I will follow the usage of the APIs. CLE: https://quic.github.io/aimet-pages/AimetDocs/api_docs/torch_cross_layer_equalization.html#api-torch-cle BC: https://quic.github.io/aimet-pages/AimetDocs/api_docs/torch_bias_correction.html#api-torch-bias-correction

[quic-mangal]

@YouYueHuang, you will also need to use the https://quic.github.io/aimet-pages/AimetDocs/api_docs/torch_quantsim.html#api-torch-quantsim API for quantizing the model. As @quic-akhobare mentioned they should be well explained with an example in the documentation, but if you get stuck, please let us know.

[YouYueHuang]

@quic-mangal, I am stuck in the step of AIMET Installation. I have no Nvidia GPU card, so I used google colab and followed the guide https://github.com/quic/aimet/blob/develop/packaging/google_colab_install.md. I succeeded in AIMET Installation and used aimet package last week. But later I repeated the installation steps (it seems the installed AIMET packages will lose once colab session is idel for several minutes), colab runtime said some package was not found.

Also, when I performed test scripts

%cd /content/aimet_code/build/
!ctest

The test results were different from the same installation steps. result 1: result 2:

In the last two cells of the following colab notebook, it shows I successfully applied CLE API to a pytorch pretrained model (all 8 tests passed). https://colab.research.google.com/drive/1eg5PmrIH4xyNuK7SbYoDRvpVILMxu6VA#scrollTo=D3ejBae0UFgK

May I know how you prepare the pakcages to make the test reproducible each time?

[quic-sendilk]

@YouYueHuang, yes, google collab will reset to factory default after prolong inactivity unless its a corporate account.

We are not observing any flakiness with the acceptance test with current develop branch, as a standalone sanity test, you could try running the release version('1.13.0') to isolate the origin of failure by cloning the repo as shown below

!git clone https://github.com/quic/aimet.git --branch 1.13.0

[quic-ssiddego]

Hi @YouYueHuang Just wanted to follow-up on this. Do you have any code ready that needs a review or have further questions for us. Please let us know if you need any help from us. Look forward to your contribution to AIMET. Thank you

[Dennis-Johnson]

Hi, could this issue be marked closed now? Saw this example for CLE and BC with a torchvision ResNet model.

[quic-akhobare]

There is an existing example now. Closing the ticket.