coremltools icon indicating copy to clipboard operation
coremltools copied to clipboard

[#2452] Fix Rank Mismatch in Quantization for Conv3d Layers

Open aryanmahawar205 opened this issue 10 months ago • 2 comments

Changes

  • Modified the _construct_quantization_op method in converter.py to expand the scale tensor to match the rank of the weight tensor.
  • Added logic to handle the expansion of the zero-point tensor as well.

Testing

  • Verified the fix by quantizing a model with Conv3d layers and ensuring that the quantization process completes without errors.
  • Tested with both w8a8 and other quantization schemes to ensure compatibility.

Related Issue

Fixes #2452 - coremltools.optimize.torch.quantization fails to handle w8a8 quantization for Conv3d layers.

Checklist

  • Code changes adhere to the project's coding standards.
  • Tests have been added/updated to verify the fix.

aryanmahawar205 avatar Feb 23 '25 13:02 aryanmahawar205

@junpeiz can you have a look at my PR. Thanks!

aryanmahawar205 avatar Feb 23 '25 13:02 aryanmahawar205

There seems to be a lot of formatting changes in this pull request. I don't think we want those changes. Please revert the formatting changes and just include your fix.

TobyRoseman avatar Feb 24 '25 21:02 TobyRoseman