Sparsebit icon indicating copy to clipboard operation
Sparsebit copied to clipboard

Add DETR Example

Open Jiang-Stan opened this issue 2 years ago • 0 comments

DETR model from: https://github.com/facebookresearch/detr

bit weight observer feature observer mAP AP50 AP75 remarks
float - - 0.421 0.623 0.443 baseline
8w8f minmax minmax 0.280 0.519 0.260
8w8f minmax minmax 0.355 0.574 0.363 aciq laplace observer for last 2 bbox embed layer weights
8w8f minmax minmax 0.358 0.576 0.365 float weight
8w8f minmax minmax 0.415 0.621 0.437 aciq laplace observer for last 2 bbox embed layer weights, float feature
8w8f minmax minmax 0.408 0.614 0.430 aciq laplace observer for last 2 bbox embed layer weights, transformer skipped for testing
8w8f minmax mse 0.386 0.595 0.400 aciq laplace observer for last 2 bbox embed layer weights
8w8f minmax mse 0.392 0.602 0.409 aciq laplace observer for last 2 bbox embed layer weights, all bmm input as float

Modifications to original model:

  • dim_t as a Parameter. For torch.fx doesn't support .to(device) in forward function. (Details)
  • Tensor[None] replaced by Tensor.unsqueeze. Unrecognized pattern. (Details)
  • Explicitly define type convertion. (Details)
  • Type convertion defined before operation. (Details)
  • remove mask-related parts in model, only batch 1 supported for inference.

Jiang-Stan avatar Sep 02 '22 08:09 Jiang-Stan