data/segtrackv2_data_utils.py does not work as expected

[GabyUnalaq]

I try to run the train.py script with SegTrack v2 and I stumbled over problems regarding the format of the database. I can understand if the database changed over the years but here it says that the last update of the database was on "December 9th, 2013" , while the code for reading the database was last updated on "June 7th, 2019".

First thing's first: The script looks for images in the GroundTruth but some of the experiments have multiple objects in motion, so the groundtruth of that experiment does not contain one image, but folders with images for every object. For instance "bmx" or "penguin" etc.

Second thing: The database contains raw images and the annotation for each of them. The problem with those is that the raw images have different extensions (example: frog.png, girl.bmp), while the annotation images have also different extensions (example: bmx.png, cheetah.bmp). The code can read only pngs while throwing an error when finding other formats. The fix would be rather simple. Old code:

# Line 58
        for exp_fname in all_exp_fnames:
            current_filenames.append(os.path.join(self.image_dirs,
                                                  experiment, exp_fname + '.png'))
            assert os.path.isfile(current_filenames[-1]), \
                "Not found image {}".format(current_filenames[-1])
            current_annotations.append(os.path.join(self.annotation_dir,
                                                    experiment, exp_fname + '.png'))
            assert os.path.isfile(current_annotations[-1]), \
                "Not found image {}".format(current_annotations[-1])
            self.samples += 1

Fixed code:

# Line 58
        for exp_fname in all_exp_fnames:
            file_name = os.path.join(self.image_dirs, experiment, exp_fname + '.png')
            if not os.path.isfile(file_name):
                file_name = os.path.join(self.image_dirs, experiment, exp_fname + '.bmp')
            assert os.path.isfile(file_name), \
                "Not found image {}".format(file_name)
            current_filenames.append(file_name)

            annot_name = os.path.join(self.annotation_dir, experiment, exp_fname + '.png')
            if not os.path.isfile(annot_name):
                annot_name = os.path.join(self.annotation_dir, experiment, exp_fname + '.bmp')
            assert os.path.isfile(annot_name), \
                "Not found image {}".format(annot_name)
            current_annotations.append(annot_name)
            self.samples += 1

Let's say you somehow dodge those problems (create an image list with the experiments with only one movable object and you update the code as above). Now a new error will be thrown at you, that I believe it also comes from the way the database is read. I have tracked the error down to adversarial_learner.py at results = sess.run(fetches, feed_dict={self.is_training: True}). The error looks like this:

Traceback (most recent call last):
  File "C:\...\Anaconda3\envs\tf37\lib\site-packages\tensorflow\python\client\session.py", line 1334, in _do_call
    return fn(*args)
  File "C:\...\Anaconda3\envs\tf37\lib\site-packages\tensorflow\python\client\session.py", line 1319, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "C:\...\Anaconda3\envs\tf37\lib\site-packages\tensorflow\python\client\session.py", line 1407, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Expected image (JPEG, PNG, or GIF), got unknown format starting with 'BM6\334\005\000\000\000\000\0006\000\000\000(\000'
	 [[{{node DecodeJpeg}}]]
	 [[{{node data_loading/IteratorGetNext}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/.../unsupervised_detection-master/train.py", line 46, in <module>
    main(sys.argv)
  File "C:/.../unsupervised_detection-master/train.py", line 43, in main
    _main()
  File "C:/.../unsupervised_detection-master/train.py", line 34, in _main
    trl.train(FLAGS)
  File "C:\...\unsupervised_detection-master\models\adversarial_learner.py", line 398, in train
    feed_dict={self.is_training: True})
  File "C:\...\Anaconda3\envs\tf37\lib\site-packages\tensorflow\python\client\session.py", line 929, in run
    run_metadata_ptr)
  File "C:\...\Anaconda3\envs\tf37\lib\site-packages\tensorflow\python\client\session.py", line 1152, in _run
    feed_dict_tensor, options, run_metadata)
  File "C:\...\Anaconda3\envs\tf37\lib\site-packages\tensorflow\python\client\session.py", line 1328, in _do_run
    run_metadata)
  File "C:\...\Anaconda3\envs\tf37\lib\site-packages\tensorflow\python\client\session.py", line 1348, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Expected image (JPEG, PNG, or GIF), got unknown format starting with 'BM6\334\005\000\000\000\000\0006\000\000\000(\000'
	 [[{{node DecodeJpeg}}]]
	 [[node data_loading/IteratorGetNext (defined at C:\...\unsupervised_detection-master\data\segtrackv2_data_utils.py:216) ]]

Process finished with exit code 1

Can you help me solve this issue and update the code so that it will be able to read the SegTrackV2 properly?