Basic Keras LSTM Fails to convert and run

[teaglin]

🐞Describe the bug

• Using Keras with TensorFlow 2.3 and Coremltools 4.1
• I have tried other versions issue still exists.
• LSTM converts but shows the warning below and does not fully convert frontend.
• Attempting to run the model fails in python
• Attempting to run in Swift spits out garbage values.
• I've tried building the model on Linux as well. Issue still persists.

System environment (please complete the following information):

• coremltools version 4.1
• OS (e.g., MacOS, Linux): MacOS
• macOS version (if applicable): 11.2.3
• XCode version (if applicable):
• How you install python (anaconda, virtualenv, system): system
• python version (e.g. 3.7): 3.9
• any other relevant information:
  • e.g. keras version if using keras conversion etc. 2.3

Converting Frontend ==> MIL Ops: 0%| | 0/13 [00:00<?, ? ops/sWARNING:root:Input ls elem type unknown. Override with <class 'coremltools.converters.mil.mil.types.type_tensor.tensor.<locals>.tensor'>

File "/usr/local/lib/python3.9/site-packages/coremltools/models/model.py", line 366, in predict raise self._framework_error File "/usr/local/lib/python3.9/site-packages/coremltools/models/model.py", line 112, in _get_proxy_and_spec return (_MLModelProxy(filename, use_cpu_only), specification, None) RuntimeError: { NSLocalizedDescription = "Error in declaring network.";

To Reproduce

• If a python script can reproduce the error, please paste the code snippet

length = 128
	feature_size = 256
	x = Input(shape=(128 , 256))
	branch_1 = Bidirectional(LSTM(256, return_sequences=True))(x)
	branch_1 = Bidirectional(LSTM(128, return_sequences=True))(branch_1)
	branch_1 = Bidirectional(LSTM(64, return_sequences=True))(branch_1)
	sequence = Dense(10, activation='softmax', name='out')(branch_1)
	model = Model([x], [sequence])
	model.compile(loss={'out':'categorical_crossentropy'}, optimizer=Adam(), metrics=['accuracy'])

	coreml_model = ct.convert(model, source='tensorflow', inputs=[ct.TensorType(name="features", shape=(1,length,feature_size))])
	test = np.zeros((1,length,feature_size))
	coreml_model.predict({'features':test})

[teaglin]

Still looking for an update. Something is definitely wonky with the converter for LSTM. I had to go back to the multi-backened version of Keras with tensorflow 1.14 to get it to finally work. Comparing the converted ml models are drastically different.

The tensorflow 2+ produces this model. It's slightly larger than the correct one and produces many different layers and ops.

Whereas the multi-backend version of Keras produces this model. Why do the layers directly convert? Whereas tensorflow 2+ produces a lot more. This model is also smaller in size and works correctly.

[TobyRoseman]

@teaglin - can you give us self-contained, standalone code to reproduce this issue?

[TobyRoseman]

Since we have not received steps to reproduce this problem, I'm going to close this issue. If we get steps to reproduce the problem, I will reopen the issue.