Best way to do augmentations

[mjb-oz]

trafficstars

Hi there,

I'm revisiting Deepforest again after not using it for 18months - things have changed a lot!

This question was previously an issue about getting the GPU working, but I cracked that, so have changed it to the current question.

Have you guys built in any augmentation functionality? Eg horizontal flipping? I need to do this for my data (all the shadows in the labelled data are to the east, but lots of the area to predict were captured in the morning so have shadows to the west), and I'm just wondering if there's a nice integrated way of doing it?

Thanks in advance!

Cheers, MJB

[easz]

it is possible to pass augmentation functions as transforms parameter to deepforest constructor https://github.com/weecology/DeepForest/blob/02aac1fd8105f18e63c321be170c2c55d82a51c6/deepforest/main.py#L23

otherwise a default transformation with HorizontalFlip is used (for training) https://github.com/weecology/DeepForest/blob/02aac1fd8105f18e63c321be170c2c55d82a51c6/deepforest/dataset.py#L25

Transformation can be defined with albumentations

[mjb-oz]

Hi Easz, thanks for the response. Sorry about the delay, I'm working on two different projects currently, so am just flipping back to this one.

Just to be clear then, if I add a list of augmentations/transforms as listed in https://github.com/albumentations-team/albumentations/blob/master/albumentations/augmentations/transforms.py, then each of these transforms gets randomly applied during each training iteration?

Eg the code would then become something like (with a random collection of transforms from Albumentations/transforms.py):

m = main.deepforest(transforms=['VerticalFlip','HorizontalFlip','Normalize','Transpose'])
m.use_release()
m.to("cuda")
m.create_trainer()
m.trainer.fit(m)

Or is it more like this:

transform = A.Compose([
    A.RandomCrop(512, 512),
    A.RandomBrightnessContrast(p=0.3),
    A.HorizontalFlip(p=0.5),
])
m = main.deepforest(transforms=transform)
m.use_release()
m.to("cuda")
m.create_trainer()
m.trainer.fit(m)

[easz]

The default augmentations from deepforest looks like this

https://github.com/weecology/DeepForest/blob/02aac1fd8105f18e63c321be170c2c55d82a51c6/deepforest/dataset.py#L25-L31

Therefore I would also append ToTensorV2() and pass the same box_params. maybe someone can confirm it.

[0xSynesty]

Hello, I am trying to figure out how to implement annotations, and I tried this :

The default augmentations from deepforest looks like this

https://github.com/weecology/DeepForest/blob/02aac1fd8105f18e63c321be170c2c55d82a51c6/deepforest/dataset.py#L25-L31

Therefore I would also append ToTensorV2() and pass the same box_params. maybe someone can confirm it.

But it does not seem to work, I get an error from albumentations saying

Your 'label_fields' are not valid - them must have same names as params in dict

Did someone manage to implement custom transforms ?

There's also a question I want to ask to you @easz , with my understanding of the code, It seems that models trains only on default augmented images (ie horizontal flip), is that the case ?

Thanks :)

[taucler]

Hi @mjb-oz I'm trying to do augmentations to improve my model's training data, with Albumentations. I started by doing the same thing as you, which I suppose you got from Albumentations github :

transform = A.Compose([
    A.HorizontalFlip(p=0.5),
    A.VerticalFlip(p=0.5),
    A.RandomBrightnessContrast(p=0.5),
    A.RandomRotate90(p=0.5),
])
m = main.deepforest(transforms=transform)
m.use_release()
m.to("cuda")
m.create_trainer()
m.trainer.fit(m)

(your first option doesn't work and throw a TypeError: 'list' object is not callable)

Unfortunately, I get the following error : KeyError: 'image' and do not understand why. Did you find a way to do your augmentations since the opening of the issue ? If so, I'd be glad if you could share the solution !

[bw4sz]

I need to document this better, I see the confusion. You need to define a transform for both train and eval in the same function. So its

    def get_transform(augment):
        """This is the new transform"""
        if augment:
            transform = A.Compose([
                A.HorizontalFlip(p=0.5),
                ToTensorV2()
            ], bbox_params=A.BboxParams(format='pascal_voc',label_fields=["category_ids"]))
            
        else:
            transform = ToTensorV2()
            
        return transform

Here is a test

def test_override_transforms():
    def get_transform(augment):
        """This is the new transform"""
        if augment:
            transform = A.Compose([
                A.HorizontalFlip(p=0.5),
                ToTensorV2()
            ], bbox_params=A.BboxParams(format='pascal_voc',label_fields=["category_ids"]))
            
        else:
            transform = ToTensorV2()
            
        return transform
    
    m = main.deepforest(transforms=get_transform)
    
    csv_file = get_data("example.csv") 
    root_dir = os.path.dirname(csv_file)
    train_ds = m.load_dataset(csv_file, root_dir=root_dir, augment=True)
    
    path, image, target = next(iter(train_ds))
    assert m.transforms.__doc__ == "This is the new transform"

I will add this to docs, thanks for catching this confusion. As always, I ask users to share their opinion of the backbone model and whether it needs to be updated. How have you found its utility? Any sample images to share?

[taucler]

Thank you for your help, the augmentation seems to work now ! Regarding my project, I'm using your backbone model to detect trees in cities and suburbs on satellite imagery (Airbus PLEIADES NEO, 0.5m/px), and I'm having wonderful results on accuracy (>99%) but only ~80% on recall. I'd be glad to talk about your work and my project with you and I'll share results when I finish my internship !

[bw4sz]

great. If you can, I strongly encourage you to link to issue #350 and add your dataset for others to benefit from in the future. The backbone can only get better with contributions from others. Especially useful if you already have the data formatted. Please do share any insights into hyper parameter settings (crop patch size?) that has helped. Satellites are often asked about.

On Mon, Aug 29, 2022 at 4:14 AM taucler @.***> wrote:

Thank you for your help, the augmentation seems to work now ! Regarding my project, I'm using your backbone model to detect trees in cities and suburbs on satellite imagery (Airbus PLEIADES NEO, 0.5m/px), and I'm having wonderful results on accuracy (>99%) but only ~80% on recall. I'll share results when I finish my internship !

— Reply to this email directly, view it on GitHub https://github.com/weecology/DeepForest/issues/326#issuecomment-1230141628, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJHBLAIC4ELUWEGCWM3LQLV3SLPRANCNFSM5YMC6OUQ . You are receiving this because you modified the open/close state.Message ID: @.***>

-- Ben Weinstein, Ph.D. Research Scientist University of Florida http://benweinstein.weebly.com/