albumentations
albumentations copied to clipboard
albumentations-team
RandomSunFlare dump
🐛 Bug
To Reproduce
add function A.RandomSunFlare(p=0.2)
Steps to reproduce the behavior:
/Pytorch/lib/python3.6/site-packages/albumentations/augmentations/functional.py", line 863, in add_sun_flare cv2.circle(overlay, (x, y), rad3, (r_color, g_color, b_color), -1) cv2.error: OpenCV(4.5.4) :-1: error: (-5:Bad argument) in function 'circle'
Overload resolution failed:
- Can't parse 'center'. Sequence item with index 1 has a wrong type
- Can't parse 'center'. Sequence item with index 1 has a wrong type
Expected behavior
Environment
- Albumentations version (e.g., 0.1.8): albumentations 1.1.0
- Python version (e.g., 3.6): 3.6
- OS (e.g., Linux): linux
- How you installed albumentations (
conda,pip, source): pip - Any other relevant information:
Additional context
Having a similar issue with OpenCV.
Error Produced is:
Traceback (most recent call last):
File "c:/Users/bilal/NYU Courses/Fall 2021/02 - MSc Thesis Option/01 - Research/Yo Hou Dataset Testing/dataset.py", line 104, in <module>
image, mask = next(iter(train_ds))
File "c:/Users/bilal/NYU Courses/Fall 2021/02 - MSc Thesis Option/01 - Research/Yo Hou Dataset Testing/dataset.py", line 66, in __getitem__
augmentations = self.transform(image=image_, mask=masks_)
File "C:\ProgramData\Anaconda3\lib\site-packages\albumentations\core\composition.py", line 210, in __call__
data = t(force_apply=force_apply, **data)
File "C:\ProgramData\Anaconda3\lib\site-packages\albumentations\core\transforms_interface.py", line 97, in __call__
return self.apply_with_params(params, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\albumentations\core\transforms_interface.py", line 112, in apply_with_params
res[key] = target_function(arg, **dict(params, **target_dependencies))
File "C:\ProgramData\Anaconda3\lib\site-packages\albumentations\core\transforms_interface.py", line 241, in apply_to_mask
return self.apply(img, **{k: cv2.INTER_NEAREST if k == "interpolation" else v for k, v in params.items()})
File "C:\ProgramData\Anaconda3\lib\site-packages\albumentations\augmentations\geometric\resize.py", line 177, in apply
return F.resize(img, height=self.height, width=self.width, interpolation=interpolation)
File "C:\ProgramData\Anaconda3\lib\site-packages\albumentations\augmentations\functional.py", line 70,
in wrapped_function
result = func(img, *args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\albumentations\augmentations\geometric\functional.py", line 277, in resize
return resize_fn(img)
File "C:\ProgramData\Anaconda3\lib\site-packages\albumentations\augmentations\functional.py", line 185, in __process_fn
chunk = process_fn(chunk, **kwargs)
cv2.error: OpenCV(4.5.4) :-1: error: (-5:Bad argument) in function 'resize'
> Overload resolution failed:
> - src data type = 0 is not supported
> - Expected Ptr<cv::UMat> for argument 'src'`
Code is:
import torchvision.transforms as T
import os
from PIL import Image
from torch.utils.data import Dataset
import numpy as np
from pycocotools.coco import COCO
import torch
import cv2
import albumentations as A
from albumentations.pytorch import ToTensorV2
class YoHouDataset(Dataset):
def __init__(self, image_dir, transform = None):
self.image_dir = image_dir
self.transform = transform
self.images = os.listdir(image_dir)
self.path_annotations = os.path.join(os.getcwd(),"Flug1_105Media_coco.json")
# self.f = open(self.path_annotations, 'r')
# self.db = json.load(self.f)
self.coco=COCO(self.path_annotations)
def __len__(self):
return len(self.images)
def __getitem__(self, index):
img_path = os.path.join(self.image_dir, self.images[index])
image_id = index
image_ = np.load(img_path,allow_pickle=True)[:,:,3]
cv2.imshow('test_image',image_)
cv2.waitKey(0) #wait for a keyboard input
cv2.destroyAllWindows()
# image_ = torch.as_tensor(image_)
image_height = image_.shape[0]
image_width = image_.shape[1]
#This will create the mask
mask_ = np.zeros((image_height, image_width))
category = 0
annIds = self.coco.getAnnIds(imgIds=image_id,catIds=category)
anns = self.coco.loadAnns(annIds)
for i in range(len(anns)):
mask_ += (self.coco.annToMask(anns[i])*(i+1))
mask_ += 1
mask_ -= np.unique(mask_)[0]
obj_ids = np.unique(mask_)
obj_ids = obj_ids[1:]
masks_ = mask_ == obj_ids[:, None, None]
print(type(masks_))
# masks = torch.as_tensor(masks, dtype=torch.uint8)
print(f"Loaded Picture {index}")
if self.transform is not None:
augmentations = self.transform(image=image_, mask=masks_)
image = augmentations['image']
masks = augmentations['mask']
# image, masks = self.transform(image, masks)
return image, masks
def get_transform(train):
transforms = []
transforms.append(T.ToTensor())
if train:
transforms.append(T.RandomHorizontalFlip(0.5))
return T.Compose(transforms)
if __name__ == "__main__":
IMAGE_HEIGHT = 2400 #Originally 2400
IMAGE_WIDTH = 3200 #Originally 3200
TRAIN_IMG_DIR = "images/Flug1_105Media/"
train_transforms = A.Compose(
[
# A.Resize(height=IMAGE_HEIGHT, width=IMAGE_WIDTH),
A.Rotate(limit=35, p=1.0),
A.HorizontalFlip(p=0.5),
A.VerticalFlip(p=0.1),
A.Normalize(
mean = [0.0, 0.0, 0.0],
std = [1.0, 1.0, 1.0],
max_pixel_value = 255.0,
),
ToTensorV2(),
],
)
train_ds = YoHouDataset(
image_dir=TRAIN_IMG_DIR,
transform=train_transforms,#transform=get_transform(train=True),
)
image, mask = next(iter(train_ds))
@Tim5Tang https://www.youtube.com/watch?v=NlhUpZKmzJ4&ab_channel=crazzylearners
Although majority of the video is in Hindi, the basic gist of it is that if you input a wrong datatype into a function, it will result in the OpenCV giving an error.
In the video he starts off by inputting an image, then he instead inputs a string and he gets the same error to spit out. He then inputs the image, but changes a different variable to produce a different error.
The error we're seeing is an OpenCV error due to inputting the wrong type.
@bilal-sher-nyu ok, thanks, i will test it, may be i have input the wrong type.
the problem happens to me..how do u fix it? I am pretty sure that the input type is right
Does the problem exist on the in latest version of the library?
Try call np.ascontiguousarray for image and mask before provide them into transforms.
@Dipet I have the same the same issue as the first post:
File "/home/hoel/.local/lib/python3.10/site-packages/albumentations/augmentations/functional.py", line 979, in add_sun_flare
cv2.circle(overlay, (x, y), rad3, (r_color, g_color, b_color), -1)
cv2.error: OpenCV(4.6.0) :-1: error: (-5:Bad argument) in function 'circle'
> Overload resolution failed:
> - Can't parse 'center'. Sequence item with index 1 has a wrong type
> - Can't parse 'center'. Sequence item with index 1 has a wrong type
However I cannot make a reproducible example. During my trainings, the error appeared seemingly randomly after 40 and 28 epochs. I thought it might be a bad combination of augmentations, so I put all the probabilities at one in an attempt to reproduce it, but the error still does not appear straight away. From the error message, it's as if the x and y extremely rarely, randomly, have a wrong type like a float when creating the circle arguments (here I assume). But since the coordinates are explicitly casted to ints, I don't see how this is happening.
Environment
- Albumentations version: 1.2.1
- Python version: 3.10
- OS: Linux
- How you installed albumentations: pip
@Dipet I have the same the same issue as the first post:
File "/home/hoel/.local/lib/python3.10/site-packages/albumentations/augmentations/functional.py", line 979, in add_sun_flare cv2.circle(overlay, (x, y), rad3, (r_color, g_color, b_color), -1) cv2.error: OpenCV(4.6.0) :-1: error: (-5:Bad argument) in function 'circle' > Overload resolution failed: > - Can't parse 'center'. Sequence item with index 1 has a wrong type > - Can't parse 'center'. Sequence item with index 1 has a wrong typeHowever I cannot make a reproducible example. During my trainings, the error appeared seemingly randomly after 40 and 28 epochs. I thought it might be a bad combination of augmentations, so I put all the probabilities at one in an attempt to reproduce it, but the error still does not appear straight away. From the error message, it's as if the x and y extremely rarely, randomly, have a wrong type like a float when creating the circle arguments (here I assume). But since the coordinates are explicitly casted to ints, I don't see how this is happening.
Environment
- Albumentations version: 1.2.1
- Python version: 3.10
- OS: Linux
- How you installed albumentations: pip
So was I! The error occurs at random in the training stage.
@Dipet I wrote a script to just keep making new images until it broke printing out the center, and can say with some certainty that the problem has to do with python ints not being 64 bit, but opencv being implemented in cpp using 64 bit ints.
Full code here (repro for me took ~24 hours using this single threaded approach).
from PIL import Image
import albumentations as A
import numpy as np
class FakeDataset:
def __init__(self):
self.fake_len = 1000
self.aug = A.RandomSunFlare(p=1.0)
def __len__(self):
return self.fake_len
def __getitem__(self, idx):
image = np.array(Image.new('RGB', (311,255)))
return self.aug(image=image)
if __name__ == "__main__":
import pdb
dataset = FakeDataset()
while True:
val = dataset[0]
Patch to add debug prints
diff --git a/albumentations/augmentations/functional.py b/albumentations/augmentations/functional.py
index 700bb79..ff74ac0 100644
--- a/albumentations/augmentations/functional.py
+++ b/albumentations/augmentations/functional.py
@@ -698,6 +698,7 @@ def add_sun_flare(img, flare_center_x, flare_center_y, src_radius, src_color, ci
output = img.copy()
for (alpha, (x, y), rad3, (r_color, g_color, b_color)) in circles:
+ print(f"center: ({x},{y})")
cv2.circle(overlay, (x, y), rad3, (r_color, g_color, b_color), -1)
cv2.addWeighted(overlay, alpha, output, 1 - alpha, 0, output)
Output (clipped obviously)
...
center: (180,184)
center: (90,-8)
center: (210,248)
center: (180,184)
center: (100,137)
center: (250,-199)
center: (70,204)
center: (280,-267)
center: (240,-177)
center: (100,137)
center: (60,227)
center: (40,-1444370434)
center: (0,-2194692778)
Traceback (most recent call last):
File "test_dl.py", line 25, in <module>
val = dataset[0]
File "test_dl.py", line 17, in __getitem__
return self.aug(image=image)
File "/home/jason.rock/albumentations/albumentations/core/transforms_interface.py", line 118, in __call__
return self.apply_with_params(params, **kwargs)
File "/home/jason.rock/albumentations/albumentations/core/transforms_interface.py", line 131, in apply_with_params
res[key] = target_function(arg, **dict(params, **target_dependencies))
File "/home/jason.rock/albumentations/albumentations/augmentations/transforms.py", line 656, in apply
return F.add_sun_flare(
File "/home/jason.rock/albumentations/albumentations/augmentations/utils.py", line 107, in wrapped_function
result = func(img, *args, **kwargs)
File "/home/jason.rock/albumentations/albumentations/augmentations/functional.py", line 702, in add_sun_flare
cv2.circle(overlay, (x, y), rad3, (r_color, g_color, b_color), -1)
cv2.error: OpenCV(4.6.0) :-1: error: (-5:Bad argument) in function 'circle'
> Overload resolution failed:
> - Can't parse 'center'. Sequence item with index 1 has a wrong type
> - Can't parse 'center'. Sequence item with index 1 has a wrong type
The root cause is https://github.com/albumentations-team/albumentations/blob/master/albumentations/augmentations/transforms.py#L689
math.tan(pi/2-epsilon) --> inf
I'm not sure I understand the goal of the tan here well enough to offer a fix at the moment, but I'll see if I can put a PR together.
If I understand this correctly, basically when the angle is pi/2 or 3pi/2 the sun is directly above or below, so the sampling of centers along the x direction doesn't work. My trivial fix to clip will at least remove the crashes but will leave behind a "data" bug at samples near pi/2 and 3pi/2. I believe there might already be a bit of a data bug here because the range of tan is covered in pi, not 2pi, so I believe the current approach might be difficult to parameterize since it is double sampling the range.
A more complete fix would be to generate points from the the line defined by centroid and the angle and then sample along that line both in the positive and negative direction.
Also if anyone stumbles on this before the fix is put in, you can workaround this issue by limiting the range of the input to tan to (.25, .75) avoid the integer overflow (pick whatever arbitrary limit you want, but keep in mind tan((.25 + epsilon)*2*pi)) = O(1/epsilon))
epsilon = .001
A.RandomSunFlare(..., angle_lower=.25 + epsilon, angle_upper=.75 - epsilon, ...)