Data generator: where use data augmentation and adaptive pixels weights

[pissw2016]

Hi, I am new to python and keras, so your code really help me a lot. When I am running segmentation.ipynb I got a error in data generation when trainning , so I did some debug. The error happend during the middel of training some times at step76 some times step 99 like it is some picture error, so I stop shuffle. It was still random. error:

Epoch 1/10 429/1238 [=========>....................] - ETA: 7:43 - loss: 0.2252 - Jaccard: 0.6086 - sparse_accuracy_ignoring_last_label: 0.9520multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/home/DATA/liutian/envs/mask/lib/python3.6/multiprocessing/pool.py", line 119, in worker result = (True, func(*args, **kwds)) File "/home/DATA/liutian/envs/mask/lib/python3.6/site-packages/keras/utils/data_utils.py", line 401, in get_index return _SHARED_SEQUENCES[uid][i] File "/home/DATA/liutian/tmp/deeplab/Keras-segmentation-deeplab-v3.1-master/utils.py", line 362, in getitem class_weights = class_weight.compute_class_weight('balanced', u_classes, valid_pixels) File "/home/DATA/liutian/envs/mask/lib/python3.6/site-packages/sklearn/utils/class_weight.py", line 55, in compute_class_weight weight = recip_freq[le.transform(classes)] IndexError: arrays used as indices must be of integer (or boolean) type """

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/DATA/liutian/tmp/deeplab/Keras-segmentation-deeplab-v3.1-master/mainslic.py", line 83, in history = SegClass.train_generator(model, train_generator, valid_generator, callbacks, mp=True) File "/home/DATA/liutian/tmp/deeplab/Keras-segmentation-deeplab-v3.1-master/utils.py", line 226, in train_generator workers=workers, use_multiprocessing=mp) File "/home/DATA/liutian/envs/mask/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "/home/DATA/liutian/envs/mask/lib/python3.6/site-packages/keras/engine/training.py", line 1418, in fit_generator initial_epoch=initial_epoch) File "/home/DATA/liutian/envs/mask/lib/python3.6/site-packages/keras/engine/training_generator.py", line 181, in fit_generator generator_output = next(output_generator) File "/home/DATA/liutian/envs/mask/lib/python3.6/site-packages/keras/utils/data_utils.py", line 601, in get six.reraise(*sys.exc_info()) File "/home/DATA/liutian/envs/mask/lib/python3.6/site-packages/six.py", line 693, in reraise raise value File "/home/DATA/liutian/envs/mask/lib/python3.6/site-packages/keras/utils/data_utils.py", line 595, in get inputs = self.queue.get(block=True).get() File "/home/DATA/liutian/envs/mask/lib/python3.6/multiprocessing/pool.py", line 644, in get raise self._value IndexError: arrays used as indices must be of integer (or boolean) type

Process finished with exit code 1

Then I comment out :

valid_pixels = self.F[n][self.Y[n] != self.n_classes]  # get all pixels (bg and foregroud) that aren't void
u_classes = np.unique(valid_pixels)
class_weights = class_weight.compute_class_weight('balanced', u_classes, valid_pixels)
class_weights = {class_id: w for class_id, w in zip(u_classes, class_weights)}
if len(class_weights) == 1:  # no bg\no fg
    if 1 in u_classes:
        class_weights[0] = 0.
    else:
        class_weights[1] = 0.
elif not len(class_weights):
    class_weights[0] = 0.
    class_weights[1] = 0.

sw_valid = np.ones(y.shape)
np.putmask(sw_valid, self.Y[n] == 0, class_weights[0])  # background weights
np.putmask(sw_valid, self.F[n], class_weights[1])  # foreground wegihts
np.putmask(sw_valid, self.Y[n] == self.n_classes, 0)
self.F_SW[n] = sw_valid

and I notice that the self.SWis just in [0,1] and not a weight consider the pixels amount relative also the model only got one input layer. sample_dict = {'pred_mask': self.SW} I change 'return self.X, self.Y, sample_dict' to return self.X, self.Y

and it works. So it seems that these code is not used, right?

Data augmentation seems not working cause the step amount in an epoch equals to image amount. How can I make it working? Or it is already working and I miss it?

[pissw2016]

After debug with the full class SegmentationGenerator(Sequence): I realize that with the official VOC2012 data set. You have to not only make your own self.image_path_list and self.label_path_list. You have to use label = np.array(Image.open(label_path)) and I believe that the class 21 refers to unlabeled class. And there is DA(data augmentation ) the data weight will work as long as your label load in the range of (0,21) . with opencv.imread you will get range (0,220).

[GuardSkill]

nice job!!! same issue