Support for Conv1D Quantization

[madarax64]

Hello, It seems as though Conv1D layer quantization is not yet supported? I get an error while trying to quantize a model containing Conv1D layers..is this going to be supported at some point?

[alanchiao]

@nutsiepully : would you be able to add this?

[nutsiepully]

Sure. I'll take a look.

[madarax64]

Hello @alanchiao @nutsiepully , I wanted to ask if there was any progress on this? Thanks

[nutsiepully]

Hi @madarax64,

Haven't started working on this yet. We are planning another release in the next 1-2 months and this should be part of it.

[madarax64]

Hi @nutsiepully, Okay, great. Thanks for the prompt response. Closing this since it should be GA by the next release. Thanks!

[nutsiepully]

You're welcome.

I'm just reopening it to make sure it stays open until I actually fix it, and so I don't forget :)

[madarax64]

@nutsiepully Haha, okay :)

[madarax64]

Hi @nutsiepully I just noticed there's a new release out (v0.4.0). Has this been fixed in this new release? Thanks

[marno1d]

Hi @nutsiepully I'm also interested in Conv1D and SeparableConv1D. Thanks!

[nutsiepully]

Hi @marno1d,

I just pushed a new release yesterday which has support for SeparableConv1D. Full support for Conv1D is still pending, but you can do some work around it by using custom quantization.

[madarax64]

Hi @nutsiepully, Great work! Can you provide an example of this please? Thanks

[nutsiepully]

https://github.com/tensorflow/model-optimization/blob/48c08d13629ff062ce1720d53a035bbfa0331b83/tensorflow_model_optimization/python/core/quantization/keras/default_8bit/quantize_numerical_test.py#L112

https://github.com/tensorflow/model-optimization/blob/48c08d13629ff062ce1720d53a035bbfa0331b83/tensorflow_model_optimization/python/core/quantization/keras/default_8bit/quantize_numerical_test.py#L135

[PiotrowskiD]

Hi @marno1d,

I just pushed a new release yesterday which has support for SeparableConv1D. Full support for Conv1D is still pending, but you can do some work around it by using custom quantization.

Hi @nutsiepully , could you elaborate on the workaround for Conv1D? Also is there any, at least approximate, ETA for the Conv1D support?

[nutsiepully]

Adding support for Conv1D standalone should be fairly simple. Conv1D internally uses the Conv2D op which is basically supported through training and conversion. You just need to write a test to verify the numerics, and also update the registry entry for it.

Conv1D with BatchNorm and that handling would need some more work - The transforms would need to be updated to match Conv1D and ensure the FQ placement is correct through conversion.

I haven't found the time to dig into it further. Happy to review a PR though.

[jennakwon06]

Does the "workaround" for Conv1D involve defining a subclass of QuantizationConfig as explained in this guide?

https://www.tensorflow.org/model_optimization/guide/quantization/training_comprehensive_guide#experiment_with_quantization

[nutsiepully]

@jennakwon06 - Yes. For Conv1D, it should be the same as the Conv2D option. Internally there's just a conv2d op, no conv1d op.

[jennakwon06]

I see. So I take that as I can reuse the QuantizationConfig of Conv2D for Conv1D.

We do have Conv1D then BatchNorm though. So would QuantizationConfig of Conv1D have to account for that?

[nutsiepully]

No, it won't be able to. You'll have to write custom transforms for that, similar to the Conv2D BatchNorm transforms. And you would have to ensure that the converter is able to handle that.

[jennakwon06]

I see. Thank you for the reply.

That must be subclassing tensorflow_model_optimization.python.core.quantization.keras.graph_transformations.transforms.Transform, as done here https://github.com/tensorflow/model-optimization/blob/master/tensorflow_model_optimization/python/core/quantization/keras/default_8bit/default_8bit_transforms.py#L171 , correct? I will take a shot at implementing this.

[nutsiepully]

Yes, you can use the Conv2D one as a guide. Perhaps, even start with modifying it. Might just work with that.

Thanks!

[teijeong]

Hi @jennakwon06, can you update?

[marbortoli]

Hey, I am also having issues to quantize my model. I want to deploy on the edge tpu, so I just set up a simple timeseries model to test it out. Any feedback on that? Thanks

[LevLavy]

It would be very useful for me as well to have conv1d and also 1d pooling supported, should be very simple to implement and would save time and make the embedded solutions more complete. This makes a lot of sense as in embedded systems that needs the quantization many cases has 1d time series of sensor measurement.

[aqibsaeed]

Any updates on this?

[choesy]

It is the middle of 2022 and still no updates... :(

[nutsiepully]

@Xhark - do we have any plans to add support for this?

Others, please try the rest of the bug to use custom quantization for support.

[jjmattathil]

Any update on this?

[tng1605]

2023, is there any plan for supporting conv1D quantization...?

[choesy]

Hey, the workaround is to use 2D convolution. But keep one dimension at 1.

[EClemMarq]

Thanks, @choesy! Your workaround worked. I'll elaborate here in case it helps someone else.

You can insert Reshape() layers into the model and then replace your Conv1D or MaxPool1D layers with their 2D equivalents (which are QAT-supported).

If you've already tried the "post-training quantization" option (i.e. NOT quantization-aware training) to convert your model, you'll see that method is doing the Reshape() + 2D operation behind the scenes. The first Netron image below is from my original model:

The second is after the "post-training quantization" option. Notice the extra Reshape() layers and the ways the convolution kernels have changed.

If you can rewrite your model to use this Reshape() + 2D operation trick, then you can apply quantization-aware-training. I didn't even need to retrain my model from scratch, I just figured out how to reshape the pretrained weights as I loaded them in. It's a little painful to get all the shaping figured out, but it does allow you to move forward.