ERROR: maximum recursion depth exceeded when using linear_quantize_activations

[ktadgh]

🐞Describing the bug

I get a maximum recursion depth exceeded error when trying to quantize a transformer model using coreml.experimental.linear_quantize_activations. When I increase the recursion limit, I get

/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown

My system is an M1 Macbook, and I'm using coremltoools 8.0. Unfortunately, I cannot share the model.

Stack Trace

(direct-core) tadghk@Tadghs-MacBook-Air coreml-onnx-export % python /Users/tadghk/coreml-onnx-export/coreml_inference.py
Running activation compression pass insert_prefix_quantize_dequantize_pair: 100%|█| 4053/4053 [00:0
Running compression pass linear_quantize_activations: start calibrating 1 samples
Running compression pass linear_quantize_activations: calibration may take a while ...
Traceback (most recent call last):
  File "/Users/tadghk/coreml-onnx-export/coreml_inference.py", line 16, in <module>
    compressed_model_a8 = cto.coreml.experimental.linear_quantize_activations(
  File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_post_training_quantization.py", line 128, in linear_quantize_activations
    activation_stats = _get_activation_calibration_stats(mlmodel, sample_data)
  File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_post_training_quantization.py", line 285, in _get_activation_calibration_stats
    debugger = ModelDebugger(fpmodel)
  File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 187, in __init__
    all_outputs = self.__class__.get_all_outputs(self.block_info)
  File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 139, in get_all_outputs
    cls.populate_outputs(output_names, block_info.operations, acc)
  File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 133, in populate_outputs
    cls.populate_outputs(prev_output_names, all_operations, acc)
  File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 133, in populate_outputs
    cls.populate_outputs(prev_output_names, all_operations, acc)
  File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 133, in populate_outputs
    cls.populate_outputs(prev_output_names, all_operations, acc)
  [Previous line repeated 988 more times]
  File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 132, in populate_outputs
    prev_output_names = cls.unique(prev_output_names)
  File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 58, in unique
    return [x for x in sequence if not (x in seen or seen.add(x))]
  File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 58, in <listcomp>
    return [x for x in sequence if not (x in seen or seen.add(x))]
  File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 58, in <listcomp>
    return [x for x in sequence if not (x in seen or seen.add(x))]
RecursionError: maximum recursion depth exceeded while calling a Python object

To Reproduce

import coremltools as ct
import coremltools.optimize as cto
import numpy as np
import time
import sys
sys.setrecursionlimit(50000)
model = ct.models.MLModel('./coreml_model.mlpackage',
                          # optimization_hints = {'specializationStrategy': ct.SpecializationStrategy.FastPrediction}
                        )

activation_config = cto.coreml.OptimizationConfig(
    global_config=cto.coreml.experimental.OpActivationLinearQuantizerConfig(
        mode="linear_symmetric"
    )
)

compressed_model_a8 = cto.coreml.experimental.linear_quantize_activations(
    model, activation_config, [{'img':np.random.randn(1,13,1024,1024)}]
)

Paste Python code snippet here, complete with any required import statements.

- If the model conversion succeeds, but there is a numerical mismatch in predictions, please include the code used for comparisons.

## System environment (please complete the following information):
 - coremltools version: 8.0
 - OS (e.g. MacOS version or Linux type): MacOS

[TobyRoseman]

It's going to be difficult to help you without being able to reproduce the issue. To start with, can you please see if this is still an issue with the latest version of coremltools?