Load a model saved using keras 2.3.1 in keras 3 (containing bidirectionnal LSTM)

[tiatariene]

Hi,

I need to load an old model trained using keras 2.3 (i don't know the tensorflow version), which contains two bidirecitonnal LSTM layers, but it stops at the loading of the first layer.

Is there any hope I could still load this model using keras 3 ?

Thank you.

Here is the message error I get :

  return keras.models.load_model(path_model, compile=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dxbz2376/miniconda3/envs/main_env312/lib/python3.12/site-packages/keras/src/saving/saving_api.py", line 183, in load_model
    return legacy_h5_format.load_model_from_hdf5(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dxbz2376/miniconda3/envs/main_env312/lib/python3.12/site-packages/keras/src/legacy/saving/legacy_h5_format.py", line 133, in load_model_from_hdf5
    model = saving_utils.model_from_config(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dxbz2376/miniconda3/envs/main_env312/lib/python3.12/site-packages/keras/src/legacy/saving/saving_utils.py", line 85, in model_from_config
    return serialization.deserialize_keras_object(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dxbz2376/miniconda3/envs/main_env312/lib/python3.12/site-packages/keras/src/legacy/saving/serialization.py", line 495, in deserialize_keras_object
    deserialized_obj = cls.from_config(
                       ^^^^^^^^^^^^^^^^
  File "/home/dxbz2376/miniconda3/envs/main_env312/lib/python3.12/site-packages/keras/src/models/model.py", line 517, in from_config
    return functional_from_config(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dxbz2376/miniconda3/envs/main_env312/lib/python3.12/site-packages/keras/src/models/functional.py", line 517, in functional_from_config
    process_layer(layer_data)
  File "/home/dxbz2376/miniconda3/envs/main_env312/lib/python3.12/site-packages/keras/src/models/functional.py", line 497, in process_layer
    layer = saving_utils.model_from_config(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dxbz2376/miniconda3/envs/main_env312/lib/python3.12/site-packages/keras/src/legacy/saving/saving_utils.py", line 85, in model_from_config
    return serialization.deserialize_keras_object(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dxbz2376/miniconda3/envs/main_env312/lib/python3.12/site-packages/keras/src/legacy/saving/serialization.py", line 495, in deserialize_keras_object
    deserialized_obj = cls.from_config(
                       ^^^^^^^^^^^^^^^^
  File "/home/dxbz2376/miniconda3/envs/main_env312/lib/python3.12/site-packages/keras/src/layers/rnn/bidirectional.py", line 314, in from_config
    config["layer"] = serialization_lib.deserialize_keras_object(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dxbz2376/miniconda3/envs/main_env312/lib/python3.12/site-packages/keras/src/saving/serialization_lib.py", line 694, in deserialize_keras_object
    cls = _retrieve_class_or_fn(
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dxbz2376/miniconda3/envs/main_env312/lib/python3.12/site-packages/keras/src/saving/serialization_lib.py", line 812, in _retrieve_class_or_fn
    raise TypeError(
TypeError: Could not locate class 'LSTM'. Make sure custom classes are decorated with `@keras.saving.register_keras_serializable()`. Full object config: {'class_name': 'LSTM', 'config': {'name': 'lstm_1', 'trainable': True, 'dtype': 'float32', 'return_sequences': True, 'return_state': False, 'go_backwards': False, 'stateful': False, 'unroll': False, 'units': 64, 'activation': 'tanh', 'recurrent_activation': 'sigmoid', 'use_bias': True, 'kernel_initializer': {'class_name': 'VarianceScaling', 'config': {'scale': 1.0, 'mode': 'fan_avg', 'distribution': 'uniform', 'seed': None}}, 'recurrent_initializer': {'class_name': 'Orthogonal', 'config': {'gain': 1.0, 'seed': None}}, 'bias_initializer': {'class_name': 'Zeros', 'config': {}}, 'unit_forget_bias': True, 'kernel_regularizer': None, 'recurrent_regularizer': None, 'bias_regularizer': None, 'activity_regularizer': None, 'kernel_constraint': None, 'recurrent_constraint': None, 'bias_constraint': None, 'dropout': 0.0, 'recurrent_dropout': 0.3, 'implementation': 2}}

[mehtamansi29]

Hi @tiatariene-

Could you help me with the code snippet to reproduce the issue ?

[tiatariene]

To reproduce the issue, you will need to create two environnements using conda and install an "old" envrionnement and a "new" one.

• Old environnement :

conda create -n old_env python=3.7
conda activate old_env
conda install tensorflow=2.4
conda install keras=2.3.1
conda install numpy=1.18.5

• New environnement :

conda create -n new_env python=3.12
conda activate new_env
conda install tensorflow

Then you need to create and save a model using the old environnement

conda activate old_env

import tensorflow

inputs = tensorflow.keras.Input(shape=(25, 128))
x = tensorflow.keras.layers.Bidirectional(
    tensorflow.keras.layers.LSTM(64, return_sequences=True), name="lstm00"
)(inputs)
x = tensorflow.keras.layers.Bidirectional(
    tensorflow.keras.layers.LSTM(64, return_sequences=True), name="lstm01"
)(x)
model = tensorflow.keras.Model(inputs, x)
model.save("model.h5")

Then you try to load it in the new environnement and the bug should happen

conda activate new_env

import keras

keras.models.load_model('model.h5', compile=False)

[mehtamansi29]

Hi @tiatariene -

Thanks for the code snippet. The error you are getting because model.save() and keras.models.load_model() is no longer support in keras2. So you need to upgrade keras version from keras2 to keras3 in old_env.

In keras3, for model savng can use **keras.saving.save_model(model, filepath, overwrite=True, kwargs) and for loading model in new_env can use keras.saving.load_model(filepath, custom_objects=None, compile=True, safe_mode=True)

Here you can fine more details regarding model saving and loading in keras3.

[tiatariene]

Hi,

Thank you for your answer.

It seems that you haven't understood my problem.

To be more precise, I have a set of models that have been developped in keras 2.3 or 2.4 that I want to load in newer keras versions like keras3. I cannot retrain them using newer keras versions.

When I load a model that doesn't contain biLSTMs, the function keras.saving.load_model(filepath, custom_objects=None, compile=True, safe_mode=True) works when loading a model saved using keras 2.3.

However when I load a model that contains biLSTM it fails by printing this error, which I think is related with the fact that LSTMs weights are saved differently in newer keras versions.

  File "/home/miniconda3/envs/main_env/lib/python3.11/site-packages/keras/src/saving/saving_api.py", line 183, in load_model
    return legacy_h5_format.load_model_from_hdf5(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miniconda3/envs/main_env/lib/python3.11/site-packages/keras/src/legacy/saving/legacy_h5_format.py", line 133, in load_model_from_hdf5
    model = saving_utils.model_from_config(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miniconda3/envs/main_env/lib/python3.11/site-packages/keras/src/legacy/saving/saving_utils.py", line 85, in model_from_config
    return serialization.deserialize_keras_object(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miniconda3/envs/main_env/lib/python3.11/site-packages/keras/src/legacy/saving/serialization.py", line 495, in deserialize_keras_object
    deserialized_obj = cls.from_config(
                       ^^^^^^^^^^^^^^^^
  File "/home/miniconda3/envs/main_env/lib/python3.11/site-packages/keras/src/models/model.py", line 517, in from_config
    return functional_from_config(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miniconda3/envs/main_env/lib/python3.11/site-packages/keras/src/models/functional.py", line 517, in functional_from_config
    process_layer(layer_data)
  File "/home/miniconda3/envs/main_env/lib/python3.11/site-packages/keras/src/models/functional.py", line 497, in process_layer
    layer = saving_utils.model_from_config(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miniconda3/envs/main_env/lib/python3.11/site-packages/keras/src/legacy/saving/saving_utils.py", line 85, in model_from_config
    return serialization.deserialize_keras_object(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miniconda3/envs/main_env/lib/python3.11/site-packages/keras/src/legacy/saving/serialization.py", line 495, in deserialize_keras_object
    deserialized_obj = cls.from_config(
                       ^^^^^^^^^^^^^^^^
  File "/home/dxbz2376/miniconda3/envs/main_env/lib/python3.11/site-packages/keras/src/layers/rnn/bidirectional.py", line 314, in from_config
    config["layer"] = serialization_lib.deserialize_keras_object(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miniconda3/envs/main_env/lib/python3.11/site-packages/keras/src/saving/serialization_lib.py", line 694, in deserialize_keras_object
    cls = _retrieve_class_or_fn(
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miniconda3/envs/main_env/lib/python3.11/site-packages/keras/src/saving/serialization_lib.py", line 812, in _retrieve_class_or_fn
    raise TypeError(
TypeError: Could not locate class 'LSTM'. Make sure custom classes are decorated with `@keras.saving.register_keras_serializable()`. Full object config: {'class_name': 'LSTM', 'config': {'name': 'lstm_1', 'trainable': True, 'dtype': 'float32', 'return_sequences': True, 'return_state': False, 'go_backwards': False, 'stateful': False, 'unroll': False, 'units': 64, 'activation': 'tanh', 'recurrent_activation': 'sigmoid', 'use_bias': True, 'kernel_initializer': {'class_name': 'VarianceScaling', 'config': {'scale': 1.0, 'mode': 'fan_avg', 'distribution': 'uniform', 'seed': None}}, 'recurrent_initializer': {'class_name': 'Orthogonal', 'config': {'gain': 1.0, 'seed': None}}, 'bias_initializer': {'class_name': 'Zeros', 'config': {}}, 'unit_forget_bias': True, 'kernel_regularizer': None, 'recurrent_regularizer': None, 'bias_regularizer': None, 'activity_regularizer': None, 'kernel_constraint': None, 'recurrent_constraint': None, 'bias_constraint': None, 'dropout': 0.0, 'recurrent_dropout': 0.3, 'implementation': 2}}

The end of the messages says in particular :

Could not locate class 'LSTM'. Make sure custom classes are decorated with @keras.saving.register_keras_serializable(). Full object config: {'class_name': 'LSTM', 'config': {'name': 'lstm_1', 'trainable': True, 'dtype': 'float32', 'return_sequences': True, 'return_state': False, 'go_backwards': False, 'stateful': False, 'unroll': False, 'units': 64, 'activation': 'tanh', 'recurrent_activation': 'sigmoid', 'use_bias': True, 'kernel_initializer': {'class_name': 'VarianceScaling', 'config': {'scale': 1.0, 'mode': 'fan_avg', 'distribution': 'uniform', 'seed': None}}, 'recurrent_initializer': {'class_name': 'Orthogonal', 'config': {'gain': 1.0, 'seed': None}}, 'bias_initializer': {'class_name': 'Zeros', 'config': {}}, 'unit_forget_bias': True, 'kernel_regularizer': None, 'recurrent_regularizer': None, 'bias_regularizer': None, 'activity_regularizer': None, 'kernel_constraint': None, 'recurrent_constraint': None, 'bias_constraint': None, 'dropout': 0.0, 'recurrent_dropout': 0.3, 'implementation': 2}}

This points to the fact keras3 load_model doesn't recognize the LSTM layer saved using keras2 save_model.

As I said, I cannot retrain this model trained in keras 2.3 and containing biLSTMs. But I would like to load it using keras3.

I hope you understand my problem better. Thank you

[mehtamansi29]

Hi @tiatariene -

You can covert .h5 model into .pb in tensorflow2.4 old_env environment.

import tensorflow as tf
from tensorflow.keras.models import load_model

model= load_model('model.h5')
def freeze_graph(graph,session,output_node_names,save_pb_dir='.',save_pb_name='model.pb',save_pb_as_text=False):
    with graph.as_default():
        output_graph_def= tf.graph.util.convert_variables_to_constants(session,grapg.as_graph_def(),output_node_names)
        tf.graph.io_write_graph(output_graph_def,save_pb_dir, save_pb_name, as_text=save_pb_as_text)

model.save('model_freeze.pb')
print('Model converted into PB successfully')

And in tensorflow2.16 new_env load that .pb model.

import tensorflow as tf
import keras

model= keras.layers.TFSMLayer('model.pb', call_endpoint='serving_default')

print('Model Load successfully')

By doing this way you can load you trained model(on old_env) can load in new_env.

[github-actions[bot]]

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed because it has been inactive for 28 days. Please reopen if you'd like to work on this further.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No

[newresu]

@mehtamansi29 note that OP wants to re-train it in Keras 3. So it wants to reload a model trained in Keras 2.3 to Keras 3.

Maybe tf should have also changed the version to Tensorflow 3.

I don't want to waste your time @tiatariene but I've tested this procedure (and seems to work?)

import keras

inputs = keras.Input(shape=(25, 128))
x = keras.layers.Bidirectional(
    keras.layers.LSTM(64, return_sequences=True), name="lstm00"
)(inputs)
x = keras.layers.Bidirectional(
    keras.layers.LSTM(64, return_sequences=True), name="lstm01"
)(x)
model = keras.Model(inputs, x)
keras.saving.load_weights(model,filepath="/content/model.h5")
model.summary()

I could be doing something wrong, but I guess is quick and worth trying.

Note that I had to use 2.8 since I didn't change the python version in colab. But here is how I saved it from 2.8 and here is how I loaded it with Keras 3

Could be good if you report fail/success in case one day someone makes a guide for these problems (load model trained in Keras2 to Keras3 to retrain.)