Sparsebit

Sparsebit copied to clipboard

中文版

Introduction

Sparsebit is a toolkit with pruning and quantization capabilities. It is designed to help researchers compress and accelerate neural network models by modifying only a few codes in existing pytorch project.

Quantization

Quantization turns full-precision params into low-bit precision params, which can compress and accelerate the model without changing its structure. This toolkit supports two common quantization paradigms, Post-Training-Quantization and Quantization-Aware-Training, with following features:

• Benefiting from the support of torch.fx, Sparsebit operates on a QuantModel, and each operation becomes a QuantModule.
• Sparsebit can easily be extended by users to accommodate their own researches. Users can register to extend important objects such as QuantModule, Quantizer and Observer by themselves.
• Exporting QDQ-ONNX is supported, which can be loaded and deployed by backends such as TensorRT and OnnxRuntime.

Results

• PTQ results on ImageNet-1k: link
• QAT results on ImageNet-1k: link

Pruning

About to released.

Resources

Documentations

Detailed usage and development guidance is located in the document. Refer to: docs

CV-Master

• We maintain a public course on quantification at Bilibili, introducing the basics of quantification and our latest work. Interested users can join the course.video
• Aiming at better enabling users to understand and apply the knowledge related to model compression, we designed related homework based on Sparsebit. Interested users can complete it by themselves.quantization_homework

Plan to re-implement

• [ ] https://github.com/megvii-research/SSQL-ECCV2022
• [ ] https://github.com/megvii-research/FQ-ViT

Join Us

• Welcome to be a member (or an intern) of our team if you are interested in Quantization, Pruning, Distillation, Self-Supervised Learning and Model Deployment.
• Submit your resume to: [email protected]

Acknowledgement

Sparsebit was inspired by several open source projects. We are grateful for these excellent projects and list them as follows:

• torch
• pytorch-quantization
• PPQ
• MQBench

License

Sparsebit is released under the Apache 2.0 license.