Augment the whole batch in the same way

[JoshuaPiinRueyPan]

Hi, I'm designing a video classification project, and want to apply your project to augment the images. In this circumstance, I want the whole batch (which means the frames in the same video) to be augmented in the same way.
I found that there's a function: Augmenter.to_deterministic(). However, it does not work as I expected... Following is my code:

def DataAugment(batchImages):
        aug = iaa.Sequential([  
                                iaa.Fliplr(0.5),
                                iaa.Crop(percent=(0, 0.1)),
                            ])

        augDet = aug.to_deterministic()
        return augDet.augment_images(batchImages)

and call such function by:

if __name__ == '__main__':
        for eachVideoPath in LIST_OF_VIDEOS:
                batchOfImagesRGB = skvideo.io.vread(eachVideoPath)
                batchOfAugmentedImages = DataAugment(batchOfImagesRGB)
                ......

The result is not effected by the 'to_deterministic()' (or even I set the arguments in Sequential() ). PS. I have print each Augmenter, and thier 'self.deterministic = True' for sure. However, the it seems randomly augment each image in the same batch...

Thanks for help!

[JoshuaPiinRueyPan]

I got it! The Augmenter.to_deterministic() does not tend to be used inside that batch. Instead, it's used between two batches. Therefore, what I need can be implemented by:

def DataAugment(batchImages):
        aug = iaa.Sequential([  
                                iaa.Fliplr(0.5),
                                iaa.Crop(percent=(0, 0.1)),
                            ])

        augDet = aug.to_deterministic()
        results = np.zeros(batchImages.shape,  batchImages.dtype)
        for i in range(batchImages.shape[0])
            results[i] = augDet.augment_image(batchImages[i])
        return results

However, it's kind of slow since I traverse the whole batch, is there any better way?

Thanks.

[arash-ashra]

Hi I'm looking for a faster way too. Anybody?

[Haetsal-Lee]

The minimum modification for a faster way I can think of is that changing the behavior of RNG (Random Number Generator) and utilizing the batch processing feature of Imgaug.

If you dig into the codes, you can find that for each batch, most of but not all augmenters sample random number required to process a batch, and do its job.

# aug =iaa.Sequential(...)
aug.random_state.generator = Generator_with_repeat(SFC64(123))
for i_child, child in enumerate(aug.get_all_children(True)):
  child.random_state.generator = Generator_with_repeat(SFC64(i_child))

I tried to make RNG to sample same number within each batch. But as I mentioned above, not all augmenters use RNG in the same way.

[thauptmann]

I used np.moveaxis() as workaround to pretend every frame is a different channel. It is faster than the deterministic trick. I am using gray images.

augmented_video = seq(image=np.moveaxis(video, 0, -1))
augmented_videos.append(np.moveaxis(augmented_video, -1, 0))