coremltools
coremltools copied to clipboard
[#2452] Fix Rank Mismatch in Quantization for Conv3d Layers
Changes
- Modified the _construct_quantization_op method in converter.py to expand the scale tensor to match the rank of the weight tensor.
- Added logic to handle the expansion of the zero-point tensor as well.
Testing
- Verified the fix by quantizing a model with Conv3d layers and ensuring that the quantization process completes without errors.
- Tested with both w8a8 and other quantization schemes to ensure compatibility.
Related Issue
Fixes #2452 - coremltools.optimize.torch.quantization fails to handle w8a8 quantization for Conv3d layers.
Checklist
- Code changes adhere to the project's coding standards.
- Tests have been added/updated to verify the fix.
@junpeiz can you have a look at my PR. Thanks!
There seems to be a lot of formatting changes in this pull request. I don't think we want those changes. Please revert the formatting changes and just include your fix.