ImageAI
ImageAI copied to clipboard
OlafenwaMoses
Cannot find images while training custom object detection
I am currently trying to train a dataset on Google Colab and have the following dependencies installed: tensorflow 1.40, tensorflow-gpu 1.13.1, keras 2.2.5, most recent version of opencv-python, as well as imageai 2.1.5. I have also tried running this on tensorflow 2.0/tensorflow-gpu 1.13.1 with the same results. I have my drive mounted to the Google Colab notebook, and the system is finding the locations of the folder where the images/annotations are as well as the pretrained yolov3 model. Whenever I attempt to train, I get an error like this:
Generating anchor boxes for training images and annotation...
Average IOU for 9 anchors: 0.74
Anchor Boxes generated.
Detection configuration saved in drive/My Drive/logos/json/detection_config.json
Training on: ['chevlogo']
Training with Batch Size: 1
Number of Experiments: 100
Training with transfer learning from pretrained Model
/usr/local/lib/python3.6/dist-packages/keras/callbacks.py:1065: UserWarning: epsilon argument is deprecated and will be removed, use min_delta instead.
warnings.warn('epsilon argument is deprecated and '
Epoch 1/100
Cannot find Cannot find Cannot find logos\validation\images\img104.jpglogos\validation\images\img107.jpg
Cannot find logos\validation\images\img116.jpg Cannot find logos\validation\images\img111.jpg logos\validation\images\img106.jpg Cannot find Cannot find Cannot find logos\validation\images\img115.jpg logos\validation\images\img119.jpg logos\validation\images\img120.jpg Cannot find Cannot find Cannot find Cannot find logos\train\images\img49.jpglogos\train\images\img93.jpglogos\train\images\img12.jpglogos\train\images\img100.jpg
Cannot find Cannot find Cannot find logos\train\images\img96.jpglogos\train\images\img92.jpglogos\train\images\img48.jpg
Cannot find Cannot find logos\validation\images\img101.jpglogos\train\images\img28.jpg
Cannot find Cannot find logos\train\images\img91.jpg logos\train\images\img61.jpg
TypeError Traceback (most recent call last)
11 frames /usr/local/lib/python3.6/dist-packages/imageai/Detection/Custom/init.py in trainModel(self) 289 callbacks=callbacks, 290 workers=4, --> 291 max_queue_size=8 292 ) 293
/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
89 warnings.warn('Update your ' + object_name + ' call to the ' +
90 'Keras 2 API: ' + signature, stacklevel=2)
---> 91 return func(*args, **kwargs)
92 wrapper._original_function = func
93 return wrapper
/usr/local/lib/python3.6/dist-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch) 1416 use_multiprocessing=use_multiprocessing, 1417 shuffle=shuffle, -> 1418 initial_epoch=initial_epoch) 1419 1420 @interfaces.legacy_generator_methods_support
/usr/local/lib/python3.6/dist-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch) 179 batch_index = 0 180 while steps_done < steps_per_epoch: --> 181 generator_output = next(output_generator) 182 183 if not hasattr(generator_output, 'len'):
/usr/local/lib/python3.6/dist-packages/keras/utils/data_utils.py in get(self) 599 except Exception as e: 600 self.stop() --> 601 six.reraise(*sys.exc_info()) 602 603
/usr/local/lib/python3.6/dist-packages/six.py in reraise(tp, value, tb) 691 if value.traceback is not tb: 692 raise value.with_traceback(tb) --> 693 raise value 694 finally: 695 value = None
/usr/local/lib/python3.6/dist-packages/keras/utils/data_utils.py in get(self) 593 try: 594 while self.is_running(): --> 595 inputs = self.queue.get(block=True).get() 596 self.queue.task_done() 597 if inputs is not None:
/usr/lib/python3.6/multiprocessing/pool.py in get(self, timeout) 642 return self._value 643 else: --> 644 raise self._value 645 646 def _set(self, i, obj):
/usr/lib/python3.6/multiprocessing/pool.py in worker(inqueue, outqueue, initializer, initargs, maxtasks, wrap_exception) 117 job, i, func, args, kwds = task 118 try: --> 119 result = (True, func(*args, **kwds)) 120 except Exception as e: 121 if wrap_exception and func is not _helper_reraises_exception:
/usr/local/lib/python3.6/dist-packages/keras/utils/data_utils.py in get_index(uid, i)
399 The value at index i.
400 """
--> 401 return _SHARED_SEQUENCES[uid][i]
402
403
/usr/local/lib/python3.6/dist-packages/imageai/Detection/Custom/generator.py in getitem(self, idx) 71 for train_instance in self.instances[l_bound:r_bound]: 72 # augment input image and fix object's position and size ---> 73 img, all_objs = self._aug_image(train_instance, net_h, net_w) 74 75 for obj in all_objs:
/usr/local/lib/python3.6/dist-packages/imageai/Detection/Custom/generator.py in _aug_image(self, instance, net_h, net_w) 161 162 if image is None: print('Cannot find ', image_name) --> 163 image = image[:,:,::-1] # RGB image 164 165 image_h, image_w, _ = image.shape
TypeError: 'NoneType' object is not subscriptable
I cannot understand why my images cannot be found since I've followed just about every piece of advice I have seen on this topic, and am at my wit's end on this one. If anyone could help me, that would be greatly appreciated.
Same thing happens for me. I even tried removing images which imageAI can't seem to find but then I got errors with another set of images.
Quick update!! I've figured what was an issue in my case.
My annotation .xml files was not correctly aligned to my images. To be specific, my filenames was named like image.JPEG, while in .xml's filename and path tags it was specified just as image with no .jpeg extension. Renaming my images(cutting .JPEG) helped to fix an issue!
In my case, I just deleted cache file then it solved. I guess the cache file has path information before moving from other folder
How do we delete the cache file?
There is a cache folder under the dataset directory. It contains .pkl files.
I'm using colab I tried deleting them but they'll reappear again bring the same problem
I'm using colab I tried deleting them but they'll reappear again bring the same problem
Same here. Used labelstudio to label the data and exported it to VOC format. Now I have to change all the xmls adding .jpg to the filenames.
I'm using colab I tried deleting them but they'll reappear again bring the same problem
Same here. Used labelstudio to label the data and exported it to VOC format. Now I have to change all the xmls adding .jpg to the filenames.
So the thing is that the annotation xml filename element does not have the file extension (when labelled with Labelstudio). You can change all annotation xml files by hand. But in the end I made this script rewriting all annotation files.
import os
import glob
path = '/content/gdrive/MyDrive/Colab Notebooks/202112 Petproject/train'
annotationspath = "{}/annotations".format(path)
imagepath = "{}/images".format(path)
for file in glob.glob("{}/*.xml".format(annotationspath)):
print(file)
tree = et.parse(file)
filename = tree.find('.//filename').text
print("filename={}".format(filename))
basename = filename.split('.')[0]
print("basename={}".format(basename))
filename_with_extension = [f for f in os.listdir(imagepath) if f.startswith(basename)][0]
print("filename_with_extension={}".format(filename_with_extension))
tree.find('.//filename').text = filename_with_extension
tree.write(file)