Cannot convert Sentence Transformer into coreml

[hassanzadeh]

❓Question

I'm trying to convert this model: https://huggingface.co/hkunlp/instructor-large into the coreml format, however, I can't make it work, one reason is that the input and output are both dicts, can you please help me understand how that model can be stored in coreml format?

[TobyRoseman]

Yes, coremltools currently does not support PyTorch models with dictionary inputs or outputs. However it's fairly simple to workaround this limitation, just create a wrapper PyTorch model that converts to/from dictionaries.

Something like the following:

class MyWrapper(nn.Module):
    def __init__(self):
        super().__init__()
        self.baseModel = <. . . . . .>

    def forward(self, value1, value2, value3):
        baseModelInput = {'inputKey1': value1, 'inputKey2': value2, 'inputKey3': value3}
        result = self.baseModel(baseModelInput)
        return (result['outputKey1'], result['outputKey2'])

Then trace and convert an instance of your wrapper model.

[hassanzadeh]

Thanks, I still get an error, ValueError: _internal_op_tensor_inplace_fill does not support dynamic index And it does not tell me exactly which line that operation is being performed. Is there a solution to that (eg, conversion to onnx first)?

Thanks