keras-team
got ValueError when loading model that have LSTM Layer.
i found an issue when i try to load my model that has LSTM Layers. Layers used in my model are
def create_model():
model = keras.Sequential([
Input((X_train.shape[1], 1)),
Conv1D(16, 1, activation = 'relu'),
Conv1D(32, 1, activation = 'relu'),
Conv1D(64, 1, activation = 'relu'),
Conv1D(128, 1, activation = 'relu'),
Conv1D(256, 1, activation = 'relu'),
Conv1D(512, 1, activation = 'relu'),
Bidirectional(LSTM(64, return_sequences = True, activation = 'tanh')),
Bidirectional(LSTM(128, return_sequences = True, activation = 'tanh')),
Bidirectional(LSTM(256, return_sequences = True, activation = 'tanh')),
# Dense(512, activation = 'relu'),
# Dense(64, activation = 'relu'),
# Dense(32, activation = 'relu'),
Dropout(0.2),
Dense(7, activation = 'softmax')
])
model.compile(loss = 'sparse_categorical_crossentropy', optimizer = 'adam', metrics = ['accuracy',])
return model
and i save my model using model.save("model_name.keras")
and when i try to load my model, i got this error
ValueError: Layer 'lstm_cell' expected 3 variables, but received 0 variables during loading. Expected: ['kernel', 'recurrent_kernel', 'bias']
I have try to do this on tensorflow and everything works fine
i'm using keras 3.0.1 using tensorflow backend and run it on google colab.
Are you trying to save and load the model in same Version? Could you please let us know what TensorFlow version you are using?
If you could provide some sample reproducible code, it would help us to investigate on the issue further. Thanks!
update : i'm using keras 3.0.2 and it's still produce the same error. tensorflow version that i used is 2.15
sorry for the late reply. here is the code
def create_model():
model = keras.Sequential([
Input((X_train.shape[1], 1)),
Conv1D(16, 1, activation = 'relu'),
Conv1D(32, 1, activation = 'relu'),
Conv1D(64, 1, activation = 'relu'),
Conv1D(128, 1, activation = 'relu'),
Conv1D(256, 1, activation = 'relu'),
Conv1D(512, 1, activation = 'relu'),
Bidirectional(LSTM(64, return_sequences = True, activation = 'tanh')),
Bidirectional(LSTM(128, return_sequences = True, activation = 'tanh')),
Bidirectional(LSTM(256, return_sequences = True, activation = 'tanh')),
Dropout(0.2),
Dense(7, activation = 'softmax')
])
model.compile(loss = 'sparse_categorical_crossentropy', optimizer = 'adam', metrics = ['accuracy',])
return model
model = create_model()
history = model.fit(X_train, y_train, epochs = 10)
model.save("new_model.keras")
# the first one, im using this way
new_model = keras.models.load_model("new_model.keras")
# and after that i'm using the way that documented on keras 3
new_model = keras.saving.load_model("new_model.keras")
and both of them still producing the same error.
I have the same problem using tensorflow/keras 2.15.0 for a simple model.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
lstm (LSTM) (None, 1) 272
dense (Dense) (None, 1) 2
=================================================================
Total params: 274 (1.07 KB)
Trainable params: 274 (1.07 KB)
Non-trainable params: 0 (0.00 Byte)
I train it on a linux server in a docker container running Python 3.11.0rc1 (FROM tensorflow/tensorflow:2.15.0-gpu).
Saved as model.keras, I get the same error when trying to load it on my windows machine running Python 3.11.7 (expected 3 variables received 0).
Saving/loading as .h5 works.
I have the same problem using tensorflow/keras 2.15.0 for a simple model.
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= lstm (LSTM) (None, 1) 272 dense (Dense) (None, 1) 2 ================================================================= Total params: 274 (1.07 KB) Trainable params: 274 (1.07 KB) Non-trainable params: 0 (0.00 Byte)I train it on a linux server in a docker container running Python 3.11.0rc1 (
FROM tensorflow/tensorflow:2.15.0-gpu). Saved asmodel.keras, I get the same error when trying to load it on my windows machine running Python 3.11.7 (expected 3 variables received 0).Saving/loading as .h5 works.
maybe you can try to use python 3.10 ?
How do I put that into the container? The tensorflow image comes with 3.11.0rc.
Secretly I hope there'll be a new container that lets me just port the whole thing to keras3, but this is my bachelor's so I can't really wait for that, it's why I'm working with h5 for now because I see no pressing reason to figure out .keras right now.
I have the same issue and it's connected Bidirectional layers. Loading a model with a Bidirectional results in error like above
I get the same error, not using Bidirectional layers, and using a model trained with keras 2.31.1 while loading with keras 2.14.0 or 2.14.1 (did not test ^3 yet). Loading works while using keras 2.13.1.
Did some digging, and it turns out that the problems are introduced in 0454a403867cd4ba23bf2604f1feaa36c357db17. Reverting the commit made my error vanish.
In my case the keys in the hdf5-file that are part of the keras save format, the keys are like lstm, lstm_2. After the commit they'll be lstm, lstm_1. Therefore the weights cannot be found in the file when trained using keras <2.14rc0 and load using >=2.14rc0.
Thank you for the deep dive @akeeman ! This issue is tricky, since that commit 0454a40 is a actually a fix for layer index naming. The naming should be lstm, lstm_1, lstm_2, and so forth, not lstm, lstm_2, etc.
However, this fix causes backwards incompatibility for saving before 2.14. I'm brainstorming fixes to ensure the previous (erroneous) index naming can be supported upon loading without adding too much code bloat. Thank you for your patience on this!
Keras 2.15 fails to load a model exported in keras_v3 format using Keras 2.14 with the same error as the one Keras 2.14 produces when trying to load one exported using Keras 2.13. This time commit https://github.com/keras-team/keras/commit/bd465a78e6e14b129e14eafb27e7fb397cdfbb54 broke the bc.
I had the same error and I fixed it by saving the model in the .h5 format instead of the .keras format
I have same issue with: tensorflow==2.16.1 keras==3.0.5 I noticed the issue is with Bidirectional layer, without it it works:
from keras.models import Sequential
from keras.layers import LSTM, Dense, Bidirectional, Input
from keras.models import load_model, save_model
import numpy as np
data = np.random.random((100, 10, 1))
target = np.random.random((100, 1))
model = Sequential()
model.add(Input(shape=(10, 1)))
model.add(Bidirectional(LSTM(20)))
model.add(Dense(1))
model.compile(loss='mae', optimizer='adam')
model.fit(data, target, epochs=10, batch_size=1, verbose=1)
save_model(model, 'model.keras')
loaded_model = load_model('model.keras')
ValueError: A total of 1 objects could not be loaded. Example error message for object <LSTMCell name=lstm_cell, built=True>:
Layer 'lstm_cell' expected 3 variables, but received 0 variables during loading. Expected: ['kernel', 'recurrent_kernel', 'bias']
List of objects that could not be loaded:
[<LSTMCell name=lstm_cell, built=True>]
Same issue for me with tensorflow==2.16.1 keras==3.0.5 ValueError: A total of 1 objects could not be loaded. Example error message for object <LSTMCell name=lstm_cell, built=True>:
Layer 'lstm_cell' expected 3 variables, but received 0 variables during loading. Expected: ['kernel', 'recurrent_kernel', 'bias']
List of objects that could not be loaded: [<LSTMCell name=lstm_cell, built=True>]
Any movement on this item? I just ran into the same issue trying to reload a saved .keras model. I have trained the models a few times and am trying to save them to disc and reload.
Same issue with tensorflow==2.16.1 keras==3.0.5 ValueError: A total of 2 objects could not be loaded. Example error message for object <LSTMCell name=lstm_cell, built=True>:
Layer 'lstm_cell' expected 3 variables, but received 0 variables during loading. Expected: ['kernel', 'recurrent_kernel', 'bias']
List of objects that could not be loaded: [<LSTMCell name=lstm_cell, built=True>, <LSTMCell name=lstm_cell, built=True>]
Is there some fix to save the .keras file as a .h5 with this loading error and not re-training the entire model again ?
@EngMohamedSaeed2001 no, there is no fix so far, but it works on tensorflow version: 2.15.1
It's not a fix per se but if u save the model in keras file and try to load it over and over... It may show the error but the model loads onto the variables and is able to perform inference.
@EngMohamedSaeed2001 no, there is no fix so far, but it works on version: 1.15.1
I didn't find keras or tensorflow version 1.15.1
Can you specify the versions of them please?
@EngMohamedSaeed2001 sorry for typo tensorflow: 2.15.1
What about keras version?
I have same issue with: tensorflow==2.16.1, keras==3.1.1, python==3.10.13
I also noticed that the issue is with Bidirectional layer, without it it works:
class OneHotEncodingLayer(layers.Layer):
def __init__(self, depth, **kwargs):
super(OneHotEncodingLayer, self).__init__(**kwargs)
self.depth = depth
def call(self, inputs):
return tf.one_hot(tf.cast(inputs, tf.int32), depth=self.depth)
inputs = keras.Input(shape=(None,), dtype="int64")
embedded = OneHotEncodingLayer(depth=max_tokens)(inputs)
x = layers.Bidirectional(layers.LSTM(32))(embedded)
x = layers.Dropout(0.5)(x)
outputs = layers.Dense(1, activation="sigmoid")(x)
model = keras.Model(inputs, outputs)
model.compile(optimizer="rmsprop",
loss="binary_crossentropy",
metrics=["accuracy"])
callbacks = [
keras.callbacks.ModelCheckpoint("one_hot_bidir_lstm.keras",
save_best_only=True)
]
model.fit(int_train_ds, validation_data=int_val_ds, epochs=1, callbacks=callbacks)```
@nkovela1 @sachinprasadhs is there a way to manually change the config file after unzipping the .keras file to resolve this error ?
I have the same issue trying to load a model with a PyTorch backend and a Bidirectional LSTM layer using keras.models.load_model. (keras==3.1.1, python==3.10.13)
ValueError: A total of 1 objects could not be loaded. Example error message for object <LSTMCell name=lstm_cell, built=True>:
Layer 'lstm_cell' expected 3 variables, but received 0 variables during loading. Expected: ['kernel', 'recurrent_kernel', 'bias']
List of objects that could not be loaded:
[<LSTMCell name=lstm_cell, built=True>]
Also run into this problem, for now going back to 2.15.0 - thank you for pointing this out as a working version.
Same issue with bidirectional on python 3.11.8, tensorflow 2.16.1 and keras 3.1.1
