PyTorch convert function for op 'dictconstruct' not implemented

[sanexodus]

Hi

I'm trying to convert RobertaForMaskedLM into coreml model, but I encountered a runtime error : PyTorch convert function for op 'dictconstruct' not implemented. My code shows as below:

from transformers import RobertaTokenizer, RobertaForMaskedLM
import torch

tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
model = RobertaForMaskedLM.from_pretrained('roberta-base')

example_input = torch.randint(1,100,(1,512))

model.cpu()
model.eval()

traced_model = torch.jit.trace(model, example_input ,strict=False)

traced_model.eval()
traced_model.save('./model_RoBERTa.pt')

# coreml conversion
from coremltools.converters.mil import register_torch_op
from coremltools.converters.mil.frontend.torch.ops import _get_inputs
from coremltools.converters.mil.mil import Builder as mb

@register_torch_op
def type_as(context, node):
    inputs = _get_inputs(context, node)
    context.add(mb.cast(x=inputs[0], dtype='int32'), node.name)

import numpy as np

mlmodel = ct.convert(traced_model, inputs=[ct.TensorType(name="input", shape=example_input.shape, dtype=np.int32)])

Add @register_torch_op because this issue: https://github.com/apple/coremltools/issues/960, if not added, "'type_as' not implemented" error will be showed.

environment: coremltools 4.1 pytorch 1.8.1

Thanks for your help in advance.

[sven-ds]

Hi, I'm using a dictionary type in my model, and I get the same error that the coreml converter cannot handle the 'dictconstruct' op. Will this operator be supported in the future? Is there a workaround like a different container type which coreml supports?

[ArthurDevNL]

Hello, starting from transformers==4.x, the transformer models have been changed to return a class/dict instead of a tuple (https://huggingface.co/transformers/migration.html?highlight=return_dict). This was for me what was causing the op not implemented to be shown. You can try if it works by initialising your model as such: model = RobertaForMaskedLM.from_pretrained('roberta-base', return_dict=False)

[saad-palapa]

@TobyRoseman I'm a contributor to the Ludwig open-source project. We use dictionaries throughout our encoder-combiner-decoder architecture.

However, converting to CoreML is giving us issues:

...
Adding op '2436' of type const
Converting PyTorch Frontend ==> MIL Ops: 100%|████████████████████████████████████████████████████▉| 1243/1244 [00:00<00:00, 1287.45 ops/s]
the following model ops are IMPLEMENTED:
  _convolution
  adaptive_avg_pool2d
  add_
  batch_norm
  constant
  dropout_
  flatten
  linear
  listconstruct
  mul
  sigmoid
  silu_
  squeeze
  tupleconstruct
  tupleunpack
the following model ops are MISSING:
  dictconstruct
Traceback (most recent call last):
  File "/root/saad-palapa/ludwig/saad.py", line 14, in <module>
    main()
  File "/root/saad-palapa/ludwig/saad.py", line 10, in main
    exporter.export(path, export_path, "model.mlmodel")
  File "/root/saad-palapa/ludwig/ludwig/model_export/coreml_exporter.py", line 40, in export
    coreml_model = ct.convert(
  File "/root/saad-palapa/ludwig/.venv/lib/python3.10/site-packages/coremltools/converters/_converters_entry.py", line 574, in convert
    mlmodel = mil_convert(
  File "/root/saad-palapa/ludwig/.venv/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 188, in mil_convert
    return _mil_convert(model, convert_from, convert_to, ConverterRegistry, MLModel, compute_units, **kwargs)
  File "/root/saad-palapa/ludwig/.venv/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 212, in _mil_convert
    proto, mil_program = mil_convert_to_proto(
  File "/root/saad-palapa/ludwig/.venv/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 286, in mil_convert_to_proto
    prog = frontend_converter(model, **kwargs)
  File "/root/saad-palapa/ludwig/.venv/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 108, in __call__
    return load(*args, **kwargs)
  File "/root/saad-palapa/ludwig/.venv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 80, in load
    return _perform_torch_convert(converter, debug)
  File "/root/saad-palapa/ludwig/.venv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 107, in _perform_torch_convert
    raise e
  File "/root/saad-palapa/ludwig/.venv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 99, in _perform_torch_convert
    prog = converter.convert()
  File "/root/saad-palapa/ludwig/.venv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/converter.py", line 519, in convert
    convert_nodes(self.context, self.graph)
  File "/root/saad-palapa/ludwig/.venv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 86, in convert_nodes
    raise RuntimeError(
RuntimeError: PyTorch convert function for op 'dictconstruct' not implemented.

I and @skanjila would like to implement the dictconstruct operator. Can you point us to some documentation or a starter guide to help us get started.

[TobyRoseman]

@saad-palapa - that would be very helpful. I suggest you take a look at our Contribution Guidelines and our doc about Building from Source.