dat file of model cannot be loaded.

[NoteDance]

Click to expand!

Issue Type

Support

Source

source

Tensorflow Version

tf 2.9.1

Custom Code

No

OS Platform and Distribution

Windows 11

Mobile device

No response

Python version

3.7

Bazel version

No response

GCC/Compiler version

No response

CUDA/cuDNN version

No response

GPU model and memory

No response

Current Behaviour?

dat file of model cannot be loaded.

Standalone code to reproduce the issue

import tensorflow as tf
import pickle
model=tf.keras.models.Sequential([
          tf.keras.layers.Flatten(input_shape=(28, 28)),
          tf.keras.layers.Dense(128,activation='relu'),
          tf.keras.layers.Dropout(0.2),
          tf.keras.layers.Dense(10)
          ])
output_file=open('model.dat','wb')
pickle.dump(model,output_file)
output_file.close()
input_file=open('model.dat','rb')
model=pickle.load(input_file)

Relevant log output

No response

[tilakrayal]

@7NoteDancing, We can see the file was loaded and saved on the drive. As the warning mentioned, please try to compile the file manually and try to load. Kindly find the gist of it here.

[NoteDance]

@7NoteDancing, We can see the file was loaded and saved on the drive. As the warning mentioned, please try to compile the file manually and try to load. Kindly find the gist of it here.

I tried but got an error. FileNotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for ram://f8918fed-a110-45e6-b6ac-10d70c4243eb/variables/variables You may be trying to load on a different device from the computational device. Consider setting the experimental_io_device option in tf.saved_model.LoadOptions to the io_device such as '/job:localhost'.

[NoteDance]

@7NoteDancing, We can see the file was loaded and saved on the drive. As the warning mentioned, please try to compile the file manually and try to load. Kindly find the gist of it here.

I find you used version 2.8.2.

[gadagashwini]

Hi @7NoteDancing, As warning says WARNING:tensorflow:Compiled the loaded model, but the compiled metrics have yet to be built. model.compile_metrics will be empty until you train or evaluate the model..

You need compile and build model before dumping to dat file. In this case you are trying load empty file. Find below complete working code

import tensorflow as tf
import numpy as np
import pickle

X_train = np.random.random((1512,28))
y_train = np.random.random((1512,10))

dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))
train_data = dataset.shuffle(len(X_train)).batch(32)
train_data = train_data.prefetch(
        buffer_size=tf.data.experimental.AUTOTUNE)

model=tf.keras.models.Sequential([
          tf.keras.layers.Dense(15, activation=tf.nn.relu, input_shape=(28,)),
          tf.keras.layers.Flatten(),
          tf.keras.layers.Dense(128,activation='relu'),
          tf.keras.layers.Dropout(0.2),
          tf.keras.layers.Dense(10)
          ])

model.compile(optimizer='adam',
                loss=tf.keras.losses.CategoricalCrossentropy(),
                metrics=['accuracy'])

model.fit(train_data, epochs=5)

Save and Load model

output_file=open('model.dat','wb')
pickle.dump(model,output_file)
output_file.close()
input_file=open('model.dat','rb')
model=pickle.load(input_file)

INFO:tensorflow:Assets written to: ram://d669cbfa-ff3d-4408-a5b3-d7d49a52d277/assets

[NoteDance]

Hi @7NoteDancing, As warning says WARNING:tensorflow:Compiled the loaded model, but the compiled metrics have yet to be built. model.compile_metrics will be empty until you train or evaluate the model..

You need compile and build model before dumping to dat file. In this case you are trying load empty file. Find below complete working code

import tensorflow as tf
import numpy as np
import pickle

X_train = np.random.random((1512,28))
y_train = np.random.random((1512,10))

dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))
train_data = dataset.shuffle(len(X_train)).batch(32)
train_data = train_data.prefetch(
        buffer_size=tf.data.experimental.AUTOTUNE)

model=tf.keras.models.Sequential([
          tf.keras.layers.Dense(15, activation=tf.nn.relu, input_shape=(28,)),
          tf.keras.layers.Flatten(),
          tf.keras.layers.Dense(128,activation='relu'),
          tf.keras.layers.Dropout(0.2),
          tf.keras.layers.Dense(10)
          ])

model.compile(optimizer='adam',
                loss=tf.keras.losses.CategoricalCrossentropy(),
                metrics=['accuracy'])

model.fit(train_data, epochs=5)

Save and Load model

output_file=open('model.dat','wb')
pickle.dump(model,output_file)
output_file.close()
input_file=open('model.dat','rb')
model=pickle.load(input_file)

INFO:tensorflow:Assets written to: ram://d669cbfa-ff3d-4408-a5b3-d7d49a52d277/assets

I tried your code and still got the error.I use tensorflow version 2.9.1,it seems that version 2.8.2 does not have this problem. FileNotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for ram://deb4af93-749d-419a-8da7-63b8c950accb/variables/variables You may be trying to load on a different device from the computational device. Consider setting the experimental_io_device option in tf.saved_model.LoadOptions to the io_device such as '/job:localhost'.

[gadagashwini]

Hi @7NoteDancing, I tried with Tf 2.9.1 on Ubuntu 20.04. I could able to successfully save and load the model, which is saved in .dat format. Thank you!

[NoteDance]

Hi @7NoteDancing, I tried with Tf 2.9.1 on Ubuntu 20.04. I could able to successfully save and load the model, which is saved in .dat format. Thank you!

Why do I get an error?

[NoteDance]

@gadagashwini I just tried it on version 2.8.2,I still can't load.

[gadagashwini]

Hi @7NoteDancing, Can you share complete error log. Thank you!

[NoteDance]

Hi @7NoteDancing, Can you share complete error log. Thank you! Is this it? WARNING:tensorflow:Compiled the loaded model, but the compiled metrics have yet to be built. model.compile_metrics will be empty until you train or evaluate the model. INFO:tensorflow:Assets written to: ram://8ae50f21-a712-435f-af4f-3120db60cc34/assets Traceback (most recent call last):

File "", line 13, in model=pickle.load(input_file)

File "E:\AI\anaconda3\lib\site-packages\keras\saving\pickle_utils.py", line 48, in deserialize_model_from_bytecode model = save_module.load_model(temp_dir)

File "E:\AI\anaconda3\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler raise e.with_traceback(filtered_tb) from None

File "E:\AI\anaconda3\lib\site-packages\tensorflow\python\saved_model\load.py", line 916, in load_partial str(err) + "\n You may be trying to load on a different device "

FileNotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for ram://d496b31a-40d3-4cb8-9bdf-99a900dad5a2/variables/variables You may be trying to load on a different device from the computational device. Consider setting the experimental_io_device option in tf.saved_model.LoadOptions to the io_device such as '/job:localhost'.

[gadagashwini]

HI @7NoteDancing, You need to use full path to variable, make sure path shouldn’t have ; special character. Thank you!

[NoteDance]

HI @7NoteDancing, You need to use full path to variable, make sure path shouldn’t have ; special character. Thank you!

I use full path,but I can't load.

[gadagashwini]

Hi @7NoteDancing, I am unable to reproduce the reported issue. I could able to save and load model. Please find the gist here. Thank you!

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler[bot]]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No