Coremltool fails to convert the most popular face recognition (tensorflow 1.x) model to coreml

[riser101]

• Name of layer type: MobileFaceNet/MobileFaceNet/Conv2d_0/BatchNorm/cond_1/AssignMovingAvg/sub/Switch
• Is this a PyTorch or a TensorFlow layer type: TensorFlow
• Your version of coremltools: 4.1
• Your version of PyTorch/TensorFlow: 1.15.0
• Impact of supporting this layer type. Why is adding support for this layer type important? Is it necessary to support a popular model or use case? This layer is part of the most popular face recognition model developed by and maintained at https://github.com/davidsandberg/facenet. And because of not implemented later, coremltools fails to convert this popular model to coreml, thereby making it's use not possible in the apple ecosystem.

Following is the Python script (3.7.12). It converts MobileNet version of the same above quoted model into MlModel / MLPackage format. Model can be downloaded from here: https://github.com/sirius-ai/MobileFaceNet_TF/tree/master/arch/pretrained_model

import coremltools as ct

mlmodel = ct.convert(
  # download and use the following model:
  # https://github.com/sirius-ai/MobileFaceNet_TF/tree/master/arch/pretrained_model/MobileFaceNet_9925_9680.pb    
  './MobileFaceNet_9925_9680.pb',
  source='tensorflow',
  inputs=[ct.TensorType(name='phase_train', shape=(1,), dtype=bool)]
)

mlmodel.save('MobileFaceNet_9925_9680-coremltools-only-approach.mlmodel')

Error given by the above script:

user@users-MacBook-Pro-2 ~/D/h/m/all-model-types-to-coreml-conversion-scripts (feature/classifier-swift) [1]> python coremltools-only-approach.py                              (tf1-env)
Running TensorFlow Graph Passes: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00,  9.63 passes/s]
Converting Frontend ==> MIL Ops:  35%|█████████████████████████████████████▉                                                                      | 959/2732 [00:00<00:00, 6250.60 ops/s]
Traceback (most recent call last):
  File "coremltools-only-approach.py", line 7, in <module>
    inputs=[ct.TensorType(name='phase_train', shape=(1,), dtype=bool)]
  File "/usr/local/Caskroom/miniconda/base/envs/tf1-env/lib/python3.7/site-packages/coremltools/converters/_converters_entry.py", line 182, in convert
    **kwargs
  File "/usr/local/Caskroom/miniconda/base/envs/tf1-env/lib/python3.7/site-packages/coremltools/converters/mil/converter.py", line 129, in mil_convert
    ConverterRegistry, **kwargs)
  File "/usr/local/Caskroom/miniconda/base/envs/tf1-env/lib/python3.7/site-packages/coremltools/converters/mil/converter.py", line 171, in mil_convert_to_proto
    prog = frontend_converter(model, **kwargs)
  File "/usr/local/Caskroom/miniconda/base/envs/tf1-env/lib/python3.7/site-packages/coremltools/converters/mil/converter.py", line 64, in __call__
    return tf1_loader.load()
  File "/usr/local/Caskroom/miniconda/base/envs/tf1-env/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/load.py", line 79, in load
    program = self._program_from_tf_ssa()
  File "/usr/local/Caskroom/miniconda/base/envs/tf1-env/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/load.py", line 228, in _program_from_tf_ssa
    return converter.convert()
  File "/usr/local/Caskroom/miniconda/base/envs/tf1-env/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/converter.py", line 406, in convert
    self.convert_main_graph(prog, graph)
  File "/usr/local/Caskroom/miniconda/base/envs/tf1-env/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/converter.py", line 335, in convert_main_graph
    outputs = convert_graph(self.context, graph, self.outputs)
  File "/usr/local/Caskroom/miniconda/base/envs/tf1-env/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/convert_utils.py", line 180, in convert_graph
    raise NotImplementedError(msg)
NotImplementedError: Conversion for TF op 'Switch' not implemented.

name: "MobileFaceNet/MobileFaceNet/Conv2d_0/BatchNorm/cond_1/AssignMovingAvg/sub/Switch"
op: "Switch"
input: "MobileFaceNet/Conv2d_0/BatchNorm/moving_mean/read"
input: "MobileFaceNet/MobileFaceNet/Conv2d_0/BatchNorm/cond_1/pred_id"
attr {
  key: "T"
  value {
    type: DT_FLOAT
  }
}
attr {
  key: "_class"
  value {
    list {
      s: "loc:@MobileFaceNet/Conv2d_0/BatchNorm/moving_mean"
    }
  }
}
attr {
  key: "_output_shapes"
  value {
    list {
      shape {
        dim {
          size: 64
        }
      }
      shape {
        dim {
          size: 64
        }
      }
    }
  }
}

[riser101]

@TobyRoseman please help us with whats the fix for this issue.

[TobyRoseman]

You either need to stop using that TensorFlow operation in your network, or implement it in coremltools using MIL operations.

See our User Guide page about Creating Custom Ops.

Also see the following file for lots of examples of implementing TensorFlow ops in MIL: coremltools/converters/mil/frontend/tensorflow/ops.py