TypeError: descriptor 'masked_fill_' for 'torch._C._TensorBase' objects doesn't apply to a 'Proxy' object

[Francis235]

When I use aimet autoquant to quant my model, I met the following issues:

• Prepare Model Traceback (most recent call last): File "/workspace/aimet/build/staging/universal/lib/python/aimet_torch/auto_quant_v2.py", line 692, in _optimize_main fp32_model = sess.wrap(self._prepare_model)(self.fp32_model) File "/workspace/aimet/build/staging/universal/lib/python/aimet_torch/auto_quant_v2.py", line 1033, in wrapper ret = fn(*args, **kwargs) File "/workspace/aimet/build/staging/universal/lib/python/aimet_torch/auto_quant_v2.py", line 498, in _prepare_model prepared_model = prepare_model(model, **self._model_preparer_kwargs) File "/workspace/aimet/build/staging/universal/lib/python/aimet_torch/model_preparer.py", line 465, in prepare_model _trace_model(model, modules_to_exclude, module_classes_to_exclude, concrete_args) File "/workspace/aimet/build/staging/universal/lib/python/aimet_torch/model_preparer.py", line 515, in _trace_model graph = tracer.trace(model, concrete_args=concrete_args) File "/usr/local/lib/python3.8/dist-packages/torch/fx/_symbolic_trace.py", line 739, in trace (self.create_arg(fn(*args)),), File "<eval_with_key>.2", line 72, in forward module_masked_fill = self.module_masked_fill(module_add_2, eq, -10000.0); module_add_2 = eq = None File "/usr/local/lib/python3.8/dist-packages/torch/fx/_symbolic_trace.py", line 717, in module_call_wrapper return self.call_module(mod, forward, args, kwargs) File "/workspace/aimet/build/staging/universal/lib/python/aimet_torch/model_preparer.py", line 505, in call_module return super().call_module(m, forward, args, kwargs) File "/usr/local/lib/python3.8/dist-packages/torch/fx/_symbolic_trace.py", line 434, in call_module return forward(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/torch/fx/_symbolic_trace.py", line 710, in forward return _orig_module_call(mod, *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in call_impl return forward_call(*input, **kwargs) File "/workspace/aimet/build/staging/universal/lib/python/aimet_torch/elementwise_ops.py", line 60, in forward return functional(*args, **kwargs) TypeError: descriptor 'masked_fill' for 'torch._C._TensorBase' objects doesn't apply to a 'Proxy' object

My code as following: ... prepared_net_g_encoder = prepare_model(symbolic_traced_net_g_encoder)

auto_quant = AutoQuant(prepared_net_g_encoder,
                   dummy_input=(x, x_l, spk, speed),
                   data_loader=unlabeled_vits_data_loader,
                   eval_callback=eval_callback)
sim, initial_accuracy = auto_quant.run_inference()

... prepare_model() and AutoQuant() can run pass without error, the error occur at auto_quant.run_inference() I noticed that aimet code has a module named QuantizableMultiheadAttention(nn.MultiheadAttention) at /workspace/aimet/TrainingExtensions/torch/src/python/aimet_torch/transformers/activation.py, I don't know whether this code is related to my issue, I try to change masked_fill_ to masked_fill, but the error still occurs, I don't know how to do now. Any suggestion will be helpful, thank you.

[Francis235]

My code branch is 'develop', the newest commit ID is: 2444ab53f7389e

[quic-hitameht]

Hi @Francis235

Are you applying prepare_model(...) outside of AutoQuant(...) API? The reason why I am asking this because AutoQuant internally tries to prepare the model before applying PTQ techniques.