failed to create mlmodel with flexible image input/output

[baaj2109]

I try to build a model with flexible image input/output, but api looks like doesn't support to do something like this.

Trace

ValueError: Unable to determine the type of the model, i.e. the source framework. Please provide the value of argument "source", from one of ["tensorflow", "pytorch", "milinternal"]. Note that model conversion requires the source package that generates the model. Please make sure you have the appropriate version of source package installed. E.g., if you're converting model originally trained with TensorFlow 1.14, make sure you have tensorflow==1.14 installed.

To Reproduce

from tensorflow import keras
from tensorflow.keras import layers

def get_model(upscale_factor=4, channels=3):
    conv_args = {
        "activation": "relu",
        "kernel_initializer": "Orthogonal",
        "padding": "same",
    }
    inputs = keras.Input(shape=(100, 100, channels))
    x = layers.Conv2D(64, 5, **conv_args)(inputs)
    x = layers.Conv2D(64, 3, **conv_args)(x)
    x = layers.Conv2D(32, 3, **conv_args)(x)
    x = layers.Conv2D(channels * (upscale_factor ** 2), 3, **conv_args)(x)
    outputs = layers.Conv2D(3, 1, **conv_args)(x)
    return keras.Model(inputs, outputs)
model = get_model()

import coremltools
input_shape = coremltools.Shape(shape= (1, coremltools.RangeDim(lower_bound= 10, 
                                                                upper_bound = -1,
                                                                default = 100),
                                       coremltools.RangeDim(lower_bound = 10, 
                                                            upper_bound = -1, 
                                                            default = 100),
                                       3))
mlmodel = coremltools.convert(model, #outputs = [coremltools.ImageType(shape = input_shape)],
                              inputs=[coremltools.ImageType(shape = input_shape)],
                              outputs = [coremltools.ImageType()]
                             )

System environment (please complete the following information):

• coremltools version (e.g., 3.0b5): 4.1
• OS (e.g., MacOS, Linux): macOS
• macOS version (if applicable): 11.4
• XCode version (if applicable):
• How you install python (anaconda, virtualenv, system): anaconda
• python version (e.g. 3.7): 3.6.12
• Tensorflow version : 2.3.0

[TobyRoseman]

@baaj2109 - When I try to run the example code, I get the following error: NameError: name 'upscale_factor' is not defined. Please update your example code. Also what version of TensorFlow are you using?

[baaj2109]

@TobyRoseman thank you for your reply. I already update the example code and my Tensorflow version is 2.3.0

[TobyRoseman]

Thanks for the updated info.

The error is caused by the outputs parameter not the inputs parameter. Doing the following converts the model without error:

mlmodel = coremltools.convert(model, inputs=[coremltools.ImageType(shape = input_shape)])

Take a look at the docstring for the outputs parameter:

    outputs : list[str] (optional)

        TensorFlow 1 and 2:
            - The ``outputs`` parameter is optional.

            - If specified, ``outputs`` is a list of string representing node
              names.

            - If ``outputs`` is not specified, the converter infers outputs to
              all be terminal identity nodes.

        PyTorch:
            - ``outputs`` must not be specified.

This parameter should be a list of strings. It's intend to be used if you do not care about all of the outputs of your original (TensorFlow) model.

We should give a better error message here. I'll leave this issue open until we've done that.

[baaj2109]

Thank you for your help. But I got another error when I try to test the model.

from PIL import Image
import numpy as np

img = np.zeros((np.random.randint(50,500),
               np.random.randint(50,500),
               3)).astype(np.uint8)
img = Image.fromarray(img)
d= {}
d[mlmodel._spec.description.input[0].name] = img
output = mlmodel.predict(d)
print(output["Identity"].shape)

and errer message is

RuntimeError: {
    NSLocalizedDescription = "Error binding image input buffer input_14.";
}

It looks like coremltools.RangeDim not working, because when I try predict with square shape image (100, 100), model run successfully.

img = np.zeros((100,100,3)).astype(np.uint8)
img = Image.fromarray(img)
d= {}
d[mlmodel._spec.description.input[0].name] = img
output = mlmodel.predict(d)
print(output["Identity"].shape)

[aseemw]

NSLocalizedDescription = "Error binding image input buffer input_14."; : is an unfortunately an error that occurs with the combination of "image inputs + rangeDim" on macOS Big Sur. The workaround is to either use image inputs with enumerated shapes or update to mac OS Monterey, in which the issue has been fixed.