Sparsebit
Sparsebit copied to clipboard
Add bias correction
- Backend: virtual
- W: Symmetric
- ResNet18: MSE Observer
- MobilenetV2: MinMax Observer
- DeiT: MinMax Observer
- A: layerwise 8bit, MinMax Observer, Symmetric
- Eval num: 5000
Model | configs | weight granularity | top1 | top5 | reported top1(from Adaround paper) |
---|---|---|---|---|---|
ResNet18 | float | - | 69.86 | 89.4 | |
ResNet18 | 4w8f | layerwise | 20.16 | 40.12 | 23.99 |
ResNet18 | 4w8f w/ BC | layerwise | 38.04 | 62.44 | 38.87 |
ResNet18 | 4w8f | channelwise | 53.86 | 76.02 | - |
ResNet18 | 4w8f w/ BC | channelwise | 59.60 | 82.96 | - |
DeiT | float | - | 72.76 | 91.12 | - |
DeiT | 8w8f | channelwise | 71.52 | 90.76 | - |
DeiT | 8w8f w/ BC | channelwise | 71.58 | 90.42 | - |
DeiT | 4w8f | channelwise | 64.18 | 86.42 | - |
DeiT | 4w8f w/ BC | channelwise | 64.68 | 86.82 | - |
Model | configs | weight qscheme | feature qscheme | top1 | top5 |
---|---|---|---|---|---|
MobileNetV2 | float | - | - | 72.60 | 90.10 |
MobileNetV2 | 8w8f | per-tensor-symmetric | per-tensor-symmetric | 67.56 | 87.44 |
MobileNetV2 | 8w8f w/ BC | per-tensor-symmetric | per-tensor-symmetric | 70.10 | 88.98 |
MobileNetV2 | 8w8f | per-tensor-symmetric | per-tensor-affine | 68.86 | 88.46 |
MobileNetV2 | 8w8f w/ BC | per-tensor-symmetric | per-tensor-affine | 71.32 | 89.96 |
MobileNetV2 | 8w8f | per-channel-symmetric | per-tensor-symmetric | 70.50 | 89.50 |
MobileNetV2 | 8w8f w/ BC | per-channel-symmetric | per-tensor-symmetric | 70.58 | 89.44 |
MobileNetV2 | 8w8f | per-channel-symmetric | per-tensor-affine | 71.44 | 89.64 |
MobileNetV2 | 8w8f w/ BC | per-channel-symmetric | per-tensor-affine | 72.1 | 89.82 |
加个mobilenetv2的bias correction结果(8w8f)