image input + rangeDim doesn't work!

[mgstar1021]

https://github.com/xuebinqin/U-2-Net I want to support flexible input, output(with image type, not multiarray) but it doesn't work.

Trace

ValueError: Unable to map torch_upsample_bilinear to core upsample

To Reproduce

net = U2NETP(3,1) 
net.load_state_dict(torch.load('u2netp.pth', map_location='cpu'))
net.cpu()
net.eval()

example_input = torch.rand(1, 3, 320, 320) * 255
input_shape = ct.Shape(shape=(1, 3, ct.RangeDim(1, 320, default=320), ct.RangeDim(1, 320, default=320)))

traced_model = torch.jit.trace(net, example_input)
model = ct.convert(traced_model, inputs=[ct.ImageType( shape= input_shape)],  convert_to='neuralnetwork')

Here are used python model and code. u2netp.pth.zip model.zip

System environment

• coremltools version (e.g., 3.0b5): https://gitlab.com/zach_nation/coremltools/-/pipelines/339060573, 4.1, 5.0b1, 5.0b2
• OS (e.g., MacOS, Linux): Mac OS
• macOS version (if applicable): Monterey, Big Sur
• XCode version (if applicable): Xcode 13, 12
• How you install python (anaconda, virtualenv, system): system
• python version (e.g. 3.7): 3.8

[TobyRoseman]

Please provide the complete output (i.e. the full stack trace).

Also please give us complete code to reproduce the issue. How do I execute this line: net = U2NETP(3,1)?

[mgstar1021]

Here is completed code.

import torch
import os
from PIL import Image
from torchvision import transforms
import coremltools as ct
from coremltools.proto import FeatureTypes_pb2 as ft
from model import U2NETP
from model import U2NET
from coremltools.models.neural_network import flexible_shape_utils

if __name__ == '__main__':
net = U2NETP(3,1) 
net.load_state_dict(torch.load('u2netp.pth', map_location='cpu'))
net.cpu()
net.eval()

example_input = torch.rand(1, 3, 320, 320) * 255
input_shape = ct.Shape(shape=(1, 3, ct.RangeDim(1, 320, default=320), ct.RangeDim(1, 320, default=320)))

traced_model = torch.jit.trace(net, example_input)
model = ct.convert(traced_model, inputs=[ct.ImageType( shape= input_shape)],  convert_to='neuralnetwork')

Traced issue:

/usr/local/bin/python3.8 /Volumes/DATA/ConversionToCoreML/main.py
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  ../c10/core/TensorImpl.h:1156.)
  return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
[W NNPACK.cpp:79] Could not initialize NNPACK! Reason: Unsupported hardware.
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/torch/nn/functional.py:3487: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
  warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/torch/nn/functional.py:3609: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/torch/nn/functional.py:1805: UserWarning: nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.
  warnings.warn("nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.")
WARNING:root:Tuple detected at graph output. This will be flattened in the converted model.
Converting Frontend ==> MIL Ops:   2%|▏         | 27/1490 [00:00<00:05, 279.48 ops/s]
Traceback (most recent call last):
  File "/Volumes/DATA/ConversionToCoreML/main.py", line 42, in <module>
    convert()
  File "/Volumes/DATA/ConversionToCoreML/main.py", line 26, in convert
    model = ct.convert(traced_model, inputs=[ct.ImageType(shape=input_shape)], convert_to='neuralnetwork')
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/coremltools/converters/_converters_entry.py", line 175, in convert
    mlmodel = mil_convert(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/coremltools/converters/mil/converter.py", line 128, in mil_convert
    proto = mil_convert_to_proto(model, convert_from, convert_to,
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/coremltools/converters/mil/converter.py", line 171, in mil_convert_to_proto
    prog = frontend_converter(model, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/coremltools/converters/mil/converter.py", line 85, in __call__
    return load(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 83, in load
    raise e
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 73, in load
    prog = converter.convert()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/coremltools/converters/mil/frontend/torch/converter.py", line 227, in convert
    convert_nodes(self.context, self.graph)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 58, in convert_nodes
    _add_op(context, node)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 694, in max_pool2d
    _max_pool(context, node, inputs)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 669, in _max_pool
    raise ValueError("@max_pool does not support symbolic input spatial shape when ceil_mode is True")
ValueError: @max_pool does not support symbolic input spatial shape when ceil_mode is True

[mgstar1021]

Any update?

[anusha66]

Were you able to find a workaround for this?