keras-cv
keras-cv copied to clipboard
Published
20 hours ago •
keras-team
keras-team
StableDiffusion Tensorflow to TF Lite
Hi @LukeWood,
For fun, I tried converting stable Diffusion model from Tensorflow to TF lite, so that I can run it on coral/edge tpu.
I tried two approaches: I- Saved model approach: II- Go through h5
will try to document them as much as possible. (sorry in advance for the long traces)
for both:
!pip install git+https://github.com/divamgupta/stable-diffusion-tensorflow --upgrade
!pip install tensorflow tensorflow_addons ftfy --upgrade
Using !pip install --upgrade keras-cv I was not able to save the model for both.
I- Saved model approach:
- Saved the model in a directory
from stable_diffusion_tf.stable_diffusion import StableDiffusion
model = StableDiffusion(
img_height=512,
img_width=512,
)
model.diffusion_model.save('/saved_model')
- lets try to load it:
import tensorflow as tf
model2 = tf.keras.models.load_model('/saved_model')
converter = tf.lite.TFLiteConverter.from_keras_model(model2)
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
- The following error is thrown:
WARNING:tensorflow:No training configuration found in save file, so the model was *not* compiled. Compile it manually.
WARNING:absl:Found untraced functions such as dense_328_layer_call_fn, dense_328_layer_call_and_return_conditional_losses, dense_329_layer_call_fn, dense_329_layer_call_and_return_conditional_losses, group_normalization_173_layer_call_fn while saving (showing 5 of 1200). These functions will not be directly callable after loading.
INFO:tensorflow:Assets written to: /tmp/tmpicdc9dmk/assets
INFO:tensorflow:Assets written to: /tmp/tmpicdc9dmk/assets
---------------------------------------------------------------------------
ConverterError Traceback (most recent call last)
<ipython-input-15-52d23a3e5390> in <module>
2 model2 = tf.keras.models.load_model('mydata/ivo/pythalpha/saved_model')
3 converter = tf.lite.TFLiteConverter.from_keras_model(model2)
----> 4 tflite_model = converter.convert()
5 open("converted_model.tflite", "wb").write(tflite_model)
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/lite/python/lite.py in wrapper(self, *args, **kwargs)
931 def wrapper(self, *args, **kwargs):
932 # pylint: disable=protected-access
--> 933 return self._convert_and_export_metrics(convert_func, *args, **kwargs)
934 # pylint: enable=protected-access
935
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/lite/python/lite.py in _convert_and_export_metrics(self, convert_func, *args, **kwargs)
909 self._save_conversion_params_metric()
910 start_time = time.process_time()
--> 911 result = convert_func(self, *args, **kwargs)
912 elapsed_time_ms = (time.process_time() - start_time) * 1000
913 if result:
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/lite/python/lite.py in convert(self)
1340 Invalid quantization parameters.
1341 """
-> 1342 saved_model_convert_result = self._convert_as_saved_model()
1343 if saved_model_convert_result:
1344 return saved_model_convert_result
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/lite/python/lite.py in _convert_as_saved_model(self)
1322 self._convert_keras_to_saved_model(temp_dir))
1323 if self.saved_model_dir:
-> 1324 return super(TFLiteKerasModelConverterV2,
1325 self).convert(graph_def, input_tensors, output_tensors)
1326 finally:
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/lite/python/lite.py in convert(self, graph_def, input_tensors, output_tensors)
1133
1134 # Converts model.
-> 1135 result = _convert_graphdef(
1136 input_data=graph_def,
1137 input_tensors=input_tensors,
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/lite/python/convert_phase.py in wrapper(*args, **kwargs)
210 else:
211 report_error_message(str(converter_error))
--> 212 raise converter_error from None # Re-throws the exception.
213 except Exception as error:
214 report_error_message(str(error))
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/lite/python/convert_phase.py in wrapper(*args, **kwargs)
203 def wrapper(*args, **kwargs):
204 try:
--> 205 return func(*args, **kwargs)
206 except ConverterError as converter_error:
207 if converter_error.errors:
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/lite/python/convert.py in convert_graphdef(input_data, input_tensors, output_tensors, **kwargs)
791 model_flags.output_arrays.append(util.get_tensor_name(output_tensor))
792
--> 793 data = convert(
794 model_flags.SerializeToString(),
795 conversion_flags.SerializeToString(),
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/lite/python/convert.py in convert(model_flags_str, conversion_flags_str, input_data_str, debug_info_str, enable_mlir_converter)
308 for error_data in _metrics_wrapper.retrieve_collected_errors():
309 converter_error.append_error(error_data)
--> 310 raise converter_error
311
312 return _run_deprecated_conversion_binary(model_flags_str,
ConverterError: /opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: error: 'tf.Conv2D' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/opt/conda/envs/rapids/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py:1267:0: note: Error code: ERROR_NEEDS_FLEX_OPS
<unknown>:0: error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: Conv2D
Details:
tf.Conv2D(tensor<?x?x?x?xf32>, tensor<1x1x1280x1280xf32>) -> (tensor<?x?x?x1280xf32>) : {data_format = "NHWC", device = "", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = "VALID", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}
tf.Conv2D(tensor<?x?x?x?xf32>, tensor<1x1x320x320xf32>) -> (tensor<?x?x?x320xf32>) : {data_format = "NHWC", device = "", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = "VALID", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}
tf.Conv2D(tensor<?x?x?x?xf32>, tensor<1x1x640x640xf32>) -> (tensor<?x?x?x640xf32>) : {data_format = "NHWC", device = "", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = "VALID", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}
tf.Conv2D(tensor<?x?x?x?xf32>, tensor<3x3x1280x1280xf32>) -> (tensor<?x?x?x1280xf32>) : {data_format = "NHWC", device = "", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = "VALID", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}
tf.Conv2D(tensor<?x?x?x?xf32>, tensor<3x3x1280x640xf32>) -> (tensor<?x?x?x640xf32>) : {data_format = "NHWC", device = "", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = "VALID", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}
tf.Conv2D(tensor<?x?x?x?xf32>, tensor<3x3x1920x1280xf32>) -> (tensor<?x?x?x1280xf32>) : {data_format = "NHWC", device = "", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = "VALID", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}
tf.Conv2D(tensor<?x?x?x?xf32>, tensor<3x3x1920x640xf32>) -> (tensor<?x?x?x640xf32>) : {data_format = "NHWC", device = "", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = "VALID", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}
tf.Conv2D(tensor<?x?x?x?xf32>, tensor<3x3x2560x1280xf32>) -> (tensor<?x?x?x1280xf32>) : {data_format = "NHWC", device = "", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = "VALID", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}
tf.Conv2D(tensor<?x?x?x?xf32>, tensor<3x3x320x320xf32>) -> (tensor<?x?x?x320xf32>) : {data_format = "NHWC", device = "", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = "VALID", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}
tf.Conv2D(tensor<?x?x?x?xf32>, tensor<3x3x320x4xf32>) -> (tensor<?x?x?x4xf32>) : {data_format = "NHWC", device = "", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = "VALID", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}
tf.Conv2D(tensor<?x?x?x?xf32>, tensor<3x3x320x640xf32>) -> (tensor<?x?x?x640xf32>) : {data_format = "NHWC", device = "", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = "VALID", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}
tf.Conv2D(tensor<?x?x?x?xf32>, tensor<3x3x640x1280xf32>) -> (tensor<?x?x?x1280xf32>) : {data_format = "NHWC", device = "", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = "VALID", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}
tf.Conv2D(tensor<?x?x?x?xf32>, tensor<3x3x640x320xf32>) -> (tensor<?x?x?x320xf32>) : {data_format = "NHWC", device = "", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = "VALID", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}
tf.Conv2D(tensor<?x?x?x?xf32>, tensor<3x3x640x640xf32>) -> (tensor<?x?x?x640xf32>) : {data_format = "NHWC", device = "", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = "VALID", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}
tf.Conv2D(tensor<?x?x?x?xf32>, tensor<3x3x960x320xf32>) -> (tensor<?x?x?x320xf32>) : {data_format = "NHWC", device = "", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = "VALID", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}
tf.Conv2D(tensor<?x?x?x?xf32>, tensor<3x3x960x640xf32>) -> (tensor<?x?x?x640xf32>) : {data_format = "NHWC", device = "", dilations = [1, 1, 1, 1], explicit_paddings = [], padding = "VALID", strides = [1, 1, 1, 1], use_cudnn_on_gpu = true}
II- Go through h5
- Save the model with format
h5
from stable_diffusion_tf.stable_diffusion import StableDiffusion
model = StableDiffusion(
img_height=512,
img_width=512,
)
model.diffusion_model.save('./stable_diffusion.h5', save_format='h5')
- Lets try to load it
import tensorflow as tf
model2 = tf.keras.models.load_model('stable_diffusion.h5')
converter = tf.lite.TFLiteConverter.from_keras_model(model2)
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
- It seems that the
TF 2.11.0does not load h5 files anymore.
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
[<ipython-input-3-72d9a214a713>](https://localhost:8080/#) in <module>
1 import tensorflow as tf
2
----> 3 model2 = tf.keras.models.load_model('stable_diffusion.h5')
4 converter = tf.lite.TFLiteConverter.from_keras_model(model2)
5 tflite_model = converter.convert()
1 frames
[/usr/local/lib/python3.7/dist-packages/keras/saving/legacy/serialization.py](https://localhost:8080/#) in class_and_config_for_serialized_keras_object(config, module_objects, custom_objects, printable_module_name)
384 if cls is None:
385 raise ValueError(
--> 386 f"Unknown {printable_module_name}: '{class_name}'. "
387 "Please ensure you are using a `keras.utils.custom_object_scope` "
388 "and that this object is included in the scope. See "
ValueError: Unknown layer: 'UNetModel'. Please ensure you are using a `keras.utils.custom_object_scope` and that this object is included in the scope. See https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object for details.
- Therefore, uninstall
tf 2.11.0and install tf2.1.0 - Attempt to load the saved h5 file:
import tensorflow as tf
model2 = tf.keras.models.load_model('stable_diffusion.h5')
converter = tf.lite.TFLiteConverter.from_keras_model(model2)
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
The load_model throws the following error:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-5-72d9a214a713> in <module>
1 import tensorflow as tf
2
----> 3 model2 = tf.keras.models.load_model('stable_diffusion.h5')
4 converter = tf.lite.TFLiteConverter.from_keras_model(model2)
5 tflite_model = converter.convert()
1 frames
/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/keras/saving/hdf5_format.py in load_model_from_hdf5(filepath, custom_objects, compile)
164 if model_config is None:
165 raise ValueError('No model found in config file.')
--> 166 model_config = json.loads(model_config.decode('utf-8'))
167 model = model_config_lib.model_from_config(model_config,
168 custom_objects=custom_objects)
AttributeError: 'str' object has no attribute 'decode'
