TF2 Object Detect API Quantization Aware Training

[mm7721]

Great to see the Tensorflow 2 Object Detect API has been released. One feature I'm very interested in is quantization aware training (as is supported in the Tensorflow 1 version). I'm assuming it's not currently supported based on the lack of quantized models in the zoo:

https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md

If my assumption is correct, do you plan to add support in the near future?

If I'm wrong and it is supported, could you point me to the relevant documentation?

Thanks.

[saberkun]

@pengchongjin @alanchiao It is interesting to see how TF2 quantization aware training API works for OD API scale.

[mm7721]

And in the meantime, is there a recommended flow to get to a quantized TF Lite model? For example:

1. Train model (with TF2 Object Detect API)
2. Export saved model (with TF2 Obect Detect API)
3. Load saved model and convert to quantized TF Lite model (with custom script)

This could be a useful stopgap if the quantization-aware training feature won't be ready for a while.

[ItsMeTheBee]

In TF1 we were able to add

graph_rewriter {
  quantization {
    delay: 48000
    weight_bits: 8
    activation_bits: 8
  }

to the config file. It seems like this is not possible in TF2 right now - is there a suitable replacement or are you planning on adding this functionality later on?

I´m trying to run custom models on the EdgeTPU so this would be very interesting for me.

[hangrymoon01]

I am interested as well in training the model in a quantization aware way for EdgeTPU in future. Please let me know if there are any plans to support this in future.

[ChowderII]

So it seems I should revert back to TF1 if I need a quantized model ASAP?

[twkx]

I'm struggling too with this issue for days...

There's no way I can get a properly quantized model with tf2 api :(

That's really annoying.....

[xumengwei]

Guys, is it supported now? I believe it's an important feature that many people need.

[FSet89]

Any news on this?

[aviraljain99]

Any updates regarding whether Quantization Aware Training is possible for TF2 now? If it is already possible, could someone guide me as to how it can be done? Is it the same as with TF1 where you use this in the config file:

graph_rewriter { quantization { delay: 48000 weight_bits: 8 activation_bits: 8 }

Is there some guide that I can look at to find more information about how to do quantization aware training using TF2?

[twkx]

I still haven't found any way to perform Quantized Aware Training in TF2, only Post Training Quantization seems possible.

I still use TF1 because of this, that's really sad 9 months after TF2 compatibility release...

[Woodstock94]

It would be great to have QAT available. Any news on this?

[FrancescoTosetti]

Any news on this?

[tensorbuffer]

would like to use TF2 once this is resolved

[daverim]

@xhark, could you update with the latest developments?

[tensorbuffer]

@Xhark Any update? We will need to rewrite our model with TF1.x if this is not resolved. If it's not ready do you have a timeline?

[rossGardiner]

I would also like to see this feature, the full integer quantisation method is just too lossy otherwise!

[EdjeElectronics]

Any news on this? I'm also interested in QAT and pre-trained quantized models for TF2. For now, I'm sticking with TF1.

[Petros626]

Hey,

I would join this conversation and ask about the ssd mobilenet fpnlite 320x320 and 640x640 how can this models not be quantized models of its running one the Edge TPU from Google Coral, hope someone could tell me this

Solution: They use post quantization

[Vilabella]

Any news on this issue? 2023 and we still dont have quantization aware training in TFOD??

[Petros626]

Any news on this issue? 2023 and we still dont have quantization aware training in TFOD??

I also beg for rotated bounding boxes, because it's state of the art (https://github.com/tensorflow/models/issues/10893). But for sure the Quantization Aware Training is a big issue. I mean the community or either TensorFlow would have to train all the models with fake quantization (int8 weights and activations) and then publish us the typical 3 files (.meta, .data and .index), which we can use for transfer-learning. Otherwise it's depending on the model architecture, which is performance suitable for embedded devices (µC, RPi, Jetson etc.).

It's a question of who starts earlier - us or the team.

[aidansmyth95]

Any news on this???

[canseloguzz]

Any news on this??? I'm still using TF 1.15 because of this...