how to use imgaug with pytorch

[flowtcw]

I want to use imgaug with pytorch. def getitem(self, index) in torch.utils.data.Dataset can process one picture at a time, but in seq(images = images, keypoints = keypoints), I must give 4 dims (B, H, W, Channel). I want to know how to use imgaug without expansion dimension. Thank you!

[aleju]

Instead of the argument images you can also just use the singular image in seq(...), which expects a (H,W,[C]) array. keypoints should then also contain the keypoints of a single image.

[1210264601]

i use seq.augment_image(img) in pytorch's getitem method,find that it is very slow. How to use augment_images with dataloader in pytorch? thanks much.

[Andy1621]

I want to know how to use it, too!!! Thank you very much.

[EliasVansteenkiste]

You can write a wrapper dataset class and add the augmentation to a transformation class. See: torchvision.transforms.compose

[apophatique]

You can write a wrapper dataset class and add the augmentation to a transformation class. See: torchvision.transforms.compose

Please, can you give a little example? I try to do it for a 2 days and it still doesnt work (dataloader + imgaug)

[gabrieldernbach]

How about

import numpy as np
from imgaug import augmenters as iaa
from torch.utils.data import DataLoader, Dataset
from torchvision import transforms

tfs = transforms.Compose([
    iaa.Sequential([
        iaa.flip.Fliplr(p=0.5),
        iaa.flip.Flipud(p=0.5),
        iaa.GaussianBlur(sigma=(0.0, 0.1)),
        iaa.MultiplyBrightness(mul=(0.65, 1.35)),
    ]).augment_image,
    transforms.ToTensor()
])


class CustomDataset(Dataset):
    def __init__(self, n_images, n_classes, transform=None):
        self.images = np.random.randint(0, 255,
                                        (n_images, 224, 224, 3),
                                        dtype=np.uint8)
        self.targets = np.random.randn(n_images, n_classes)
        self.transform = transform

    def __getitem__(self, item):
        image = self.images[item]
        target = self.targets[item]

        if self.transform:
            image = self.transform(image)

        return image, target

    def __len__(self):
        return len(self.images)


custom_ds = CustomDataset(n_images=50, n_classes=10, transform=tfs)
custom_dl = DataLoader(custom_ds, batch_size=64,
                       num_workers=4, pin_memory=True)
print(custom_ds[3])

@aleju is this was you meant with ...?

use the singular image

Is this the recommended way? (performance-wise)

[YangZhang4065]

How about

import numpy as np
from imgaug import augmenters as iaa
from torch.utils.data import DataLoader, Dataset
from torchvision import transforms

tfs = transforms.Compose([
    iaa.Sequential([
        iaa.flip.Fliplr(p=0.5),
        iaa.flip.Flipud(p=0.5),
        iaa.GaussianBlur(sigma=(0.0, 0.1)),
        iaa.MultiplyBrightness(mul=(0.65, 1.35)),
    ]).augment_image,
    transforms.ToTensor()
])


class CustomDataset(Dataset):
    def __init__(self, n_images, n_classes, transform=None):
        self.images = np.random.randint(0, 255,
                                        (n_images, 224, 224, 3),
                                        dtype=np.uint8)
        self.targets = np.random.randn(n_images, n_classes)
        self.transform = transform

    def __getitem__(self, item):
        image = self.images[item]
        target = self.targets[item]

        if self.transform:
            image = self.transform(image)

        return image, target

    def __len__(self):
        return len(self.images)


custom_ds = CustomDataset(n_images=50, n_classes=10, transform=tfs)
custom_dl = DataLoader(custom_ds, batch_size=64,
                       num_workers=4, pin_memory=True)
print(custom_ds[3])

@aleju is this was you meant with ...?

use the singular image

Is this the recommended way? (performance-wise)

Thank you for the code. However, note that imgaug is always seeded. Your code does not initialize each dataloader worker properly (https://github.com/pytorch/pytorch/issues/5059). What would happen is that all your 4 dataloader workers will share the same augmentation sequence and every 4 images will have the same augmentation pattern. This would cause problems because your augmentation won't be random.
A fix would be:

def worker_init_fn(worker_id):
    imgaug.seed(np.random.get_state()[1][0] + worker_id)
custom_dl = DataLoader(custom_ds, batch_size=64,
                       num_workers=4, pin_memory=True, worker_init_fn=worker_init_fn)

[Ekta246]

How about

import numpy as np
from imgaug import augmenters as iaa
from torch.utils.data import DataLoader, Dataset
from torchvision import transforms

tfs = transforms.Compose([
    iaa.Sequential([
        iaa.flip.Fliplr(p=0.5),
        iaa.flip.Flipud(p=0.5),
        iaa.GaussianBlur(sigma=(0.0, 0.1)),
        iaa.MultiplyBrightness(mul=(0.65, 1.35)),
    ]).augment_image,
    transforms.ToTensor()
])


class CustomDataset(Dataset):
    def __init__(self, n_images, n_classes, transform=None):
        self.images = np.random.randint(0, 255,
                                        (n_images, 224, 224, 3),
                                        dtype=np.uint8)
        self.targets = np.random.randn(n_images, n_classes)
        self.transform = transform

    def __getitem__(self, item):
        image = self.images[item]
        target = self.targets[item]

        if self.transform:
            image = self.transform(image)

        return image, target

    def __len__(self):
        return len(self.images)


custom_ds = CustomDataset(n_images=50, n_classes=10, transform=tfs)
custom_dl = DataLoader(custom_ds, batch_size=64,
                       num_workers=4, pin_memory=True)
print(custom_ds[3])

@aleju is this was you meant with ...?

use the singular image

Is this the recommended way? (performance-wise)

Thank you for the code. However, note that imgaug is always seeded. Your code does not initialize each dataloader worker properly (pytorch/pytorch#5059). What would happen is that all your 4 dataloader workers will share the same augmentation sequence and every 4 images will have the same augmentation pattern. This would cause problems because your augmentation won't be random. A fix would be:

def worker_init_fn(worker_id):
    imgaug.seed(np.random.get_state()[1][0] + worker_id)
custom_dl = DataLoader(custom_ds, batch_size=64,
                       num_workers=4, pin_memory=True, worker_init_fn=worker_init_fn)

Very helpful post! The imgaug.Sequential() function also has this bool parameter as random_order: bool. That can help too! However, yes it is totally agreed that any transforms applied should be seeded.

Thanks again!

[HXLH50K]

How about

import numpy as np
from imgaug import augmenters as iaa
from torch.utils.data import DataLoader, Dataset
from torchvision import transforms

tfs = transforms.Compose([
    iaa.Sequential([
        iaa.flip.Fliplr(p=0.5),
        iaa.flip.Flipud(p=0.5),
        iaa.GaussianBlur(sigma=(0.0, 0.1)),
        iaa.MultiplyBrightness(mul=(0.65, 1.35)),
    ]).augment_image,
    transforms.ToTensor()
])


class CustomDataset(Dataset):
    def __init__(self, n_images, n_classes, transform=None):
        self.images = np.random.randint(0, 255,
                                        (n_images, 224, 224, 3),
                                        dtype=np.uint8)
        self.targets = np.random.randn(n_images, n_classes)
        self.transform = transform

    def __getitem__(self, item):
        image = self.images[item]
        target = self.targets[item]

        if self.transform:
            image = self.transform(image)

        return image, target

    def __len__(self):
        return len(self.images)


custom_ds = CustomDataset(n_images=50, n_classes=10, transform=tfs)
custom_dl = DataLoader(custom_ds, batch_size=64,
                       num_workers=4, pin_memory=True)
print(custom_ds[3])

@aleju is this was you meant with ...?

use the singular image

Is this the recommended way? (performance-wise)

Thanks for your code, but my task is image segmetation, and how to use the same augmentation to a pair of img&mask? By the way, my input image and mask are .npy file.

[koyakuwe]

Hi @gabrieldernbach @aleju When I try your code, I face error: ValueError: At least one stride in the given numpy array is negative, and tensors with negative strides are not currently supported. (You can probably work around this by making a copy of your array with array.copy().)

It happens at transforms.ToTensor(). My imgaug verdsion is 0.2.9.

Thanks

[PKUxxz]

Hi @gabrieldernbach @aleju When I try your code, I face error: ValueError: At least one stride in the given numpy array is negative, and tensors with negative strides are not currently supported. (You can probably work around this by making a copy of your array with array.copy().)

It happens at transforms.ToTensor(). My imgaug verdsion is 0.2.9.

Thanks

Hello, I met this problem too. However, it was fixed by arranging codes like this:

np.asarray,
seq.augment_image,
np.copy,
transforms.ToTensor(),

Note that in some cases( such as reading images via ImageFolder), there might be an error like ValueError: Expected argument 'images' to be any of the following: None or array or iterable of array. Got type: <class 'PIL.Image.Image'>.. So np.asarray is needed to transfer the type of your image. And np.copy is used to solve the error raised by the negativity of some strides.