InvalidArgumentError: Graph execution error:

[wshijiang]

Can anyone help me

import keras from keras import layers import matplotlib.pyplot as plt from keras.utils import image_dataset_from_directory ​ epochs = 30 image_size = (373, 454) ​ train_ds, val_ds = image_dataset_from_directory("/kaggle/input/fractured-images/images", validation_split=0.4, subset="both", image_size=image_size, seed=1337, batch_size=128) ​ ​ def model_function(shape): inputs = keras.Input(shape) w = layers.Rescaling(1.0 /255)(inputs) for size in [256, 512, 728, 1024]: w = layers.Conv2D(size, 3, activation="relu")(w) w = layers.MaxPooling2D(pool_size=2)(w) w = layers.Conv2D(1024, 3, activation="relu")(w) w = layers.Flatten()(w) outputs = layers.Dense(1, activation="sigmoid")(w) return keras.Model(inputs=inputs, outputs=outputs) ​ model = model_function(shape=image_size + (3,)) model.summary() keras.utils.plot_model(model, f"fractures_test_{epochs}1.png", show_shapes=True) ​ callbacks = [ keras.callbacks.ModelCheckpoint( filepath=f"fractured_model_test{epochs}_1.keras", save_best_only=True, monitor="val_loss" ) ] ​ model.compile( loss="binary_crossentropy", optimizer="rmsprop", metrics=["accuracy"] ) ​ history = model.fit( train_ds, epochs=epochs, validation_data=val_ds, callbacks=callbacks )

2024-07-17 16:02:44.093403: E tensorflow/core/lib/jpeg/jpeg_mem.cc:327] Premature end of JPEG data. Stopped at line 382/454 2024-07-17 16:02:44.133631: E tensorflow/core/lib/jpeg/jpeg_mem.cc:327] Premature end of JPEG data. Stopped at line 430/454

InvalidArgumentError Traceback (most recent call last) Cell In[16], line 48 34 callbacks = [ 35 keras.callbacks.ModelCheckpoint( 36 filepath=f"fractured_model_test_{epochs}_1.keras", (...) 39 ) 40 ] 42 model.compile( 43 loss="binary_crossentropy", 44 optimizer="rmsprop", 45 metrics=["accuracy"] 46 ) ---> 48 history = model.fit( 49 train_ds, 50 epochs=epochs, 51 validation_data=val_ds, 52 callbacks=callbacks 53 ) 55 history_dict = history.history 56 loss_valus = history_dict["loss"]

File /opt/conda/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py:122, in filter_traceback..error_handler(*args, **kwargs) 119 filtered_tb = _process_traceback_frames(e.traceback) 120 # To get the full stack trace, call: 121 # keras.config.disable_traceback_filtering() --> 122 raise e.with_traceback(filtered_tb) from None 123 finally: 124 del filtered_tb

File /opt/conda/lib/python3.10/site-packages/tensorflow/python/eager/execute.py:53, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name) 51 try: 52 ctx.ensure_initialized() ---> 53 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name, 54 inputs, attrs, num_outputs) 55 except core._NotOkStatusException as e: 56 if name is not None:

InvalidArgumentError: Graph execution error:

Detected at node decode_image/DecodeImage defined at (most recent call last): jpeg::Uncompress failed. Invalid JPEG data or crop window. [[{{node decode_image/DecodeImage}}]] [[IteratorGetNext]] [Op:__inference_one_step_on_iterator_11223] 2024-07-17 16:02:44.259599: E tensorflow/core/lib/jpeg/jpeg_mem.cc:327] Premature end of JPEG data. Stopped at line 446/454 2024-07-17 16:02:44.441775: E tensorflow/core/lib/jpeg/jpeg_mem.cc:327] Premature end of JPEG data. Stopped at line 350/454

[mehtamansi29]

Hi @iwqculrbud -

Thanks for reporting this issue. Currently I am to find the dataset directory to reproduce this issue. If possible Can you help me to with your dataset? Thanks..!!

[wshijiang]

https://kaggle.com/datasets/e26fb9d642838aaf211ab8e41238df185bb60f6199ce07eafffae86085759d89 This is an alternate address,thank you!!!

[mehtamansi29]

Hi @iwqculrbud -

Thanks for the dataset. Actually "InvalidArgumentError: Graph execution error" error cause by there were corrupt images are there in your dataset. To resolve this error remove corrupt images and apply some preprocessing will resolve this error.

Here is the solution you can try to create train_ds and val_ds and then train your model:

image_paths=[]
labels=[]
for root, dirs, filenames in os.walk(files_dir):
    for filename in filenames:
        if filename.lower().endswith((".jpg", ".jpeg")):
            if not filename.startswith('.'):
                image_path = os.path.join(root, filename)
                img = cv2.imread(image_path)
                if img is not None:
                    image_paths.append(image_path) 
                    labels.append(os.path.basename(root))

def decode_image(image_path, label):
    img = keras.utils.load_img(image_path, target_size=(224, 224))
    img = keras.utils.img_to_array(img)
    img = tf.cast(img, tf.float32) / 255.0
    return img, label

def decode_image_wrapper(image_path, label):
    img, label = tf.numpy_function(decode_image, [image_path, label], [tf.float32, tf.string])
    img.set_shape((224, 224, 3))
    label.set_shape(())
    return img, label

image_paths_ds = tf.data.Dataset.from_tensor_slices(image_paths)
labels_ds = tf.data.Dataset.from_tensor_slices(labels)
dataset = tf.data.Dataset.zip((image_paths_ds, labels_ds))
dataset = dataset.map(decode_image_wrapper, num_parallel_calls=tf.data.AUTOTUNE)
train_size = int(0.8 * len(image_paths))
val_size = len(image_paths) - train_size
train_ds = dataset.take(train_size)
val_ds = dataset.skip(train_size)

[wshijiang]

你好 -

感谢您的数据集。实际上，“InvalidArgumentError： Graph execution error”错误是由数据集中存在损坏的图像引起的。要解决此错误，请删除损坏的图像并应用一些预处理将解决此错误。

下面是您可以尝试创建train_ds和val_ds然后训练模型的解决方案：

image_paths=[]
labels=[]
for root, dirs, filenames in os.walk(files_dir):
    for filename in filenames:
        if filename.lower().endswith((".jpg", ".jpeg")):
            if not filename.startswith('.'):
                image_path = os.path.join(root, filename)
                img = cv2.imread(image_path)
                if img is not None:
                    image_paths.append(image_path) 
                    labels.append(os.path.basename(root))

def decode_image(image_path, label):
    img = keras.utils.load_img(image_path, target_size=(224, 224))
    img = keras.utils.img_to_array(img)
    img = tf.cast(img, tf.float32) / 255.0
    return img, label

def decode_image_wrapper(image_path, label):
    img, label = tf.numpy_function(decode_image, [image_path, label], [tf.float32, tf.string])
    img.set_shape((224, 224, 3))
    label.set_shape(())
    return img, label

image_paths_ds = tf.data.Dataset.from_tensor_slices(image_paths)
labels_ds = tf.data.Dataset.from_tensor_slices(labels)
dataset = tf.data.Dataset.zip((image_paths_ds, labels_ds))
dataset = dataset.map(decode_image_wrapper, num_parallel_calls=tf.data.AUTOTUNE)
train_size = int(0.8 * len(image_paths))
val_size = len(image_paths) - train_size
train_ds = dataset.take(train_size)
val_ds = dataset.skip(train_size)

hello

This doesn't work for me, thank you

[mehtamansi29]

Hi @iwqculrbud -

Your dataset images are corrupted hence, we need to do preprocessing for corrupted images. You can find attached gist with detailed solution.

[github-actions[bot]]

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

[mehtamansi29]

Hi @iwqculrbud -

Does the issue is still reproduce to you ?

[github-actions[bot]]

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed because it has been inactive for 28 days. Please reopen if you'd like to work on this further.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No