albumentations
albumentations copied to clipboard
albumentations-team
Implementation of #617. `check_validity` BBox parameter
Fix #617
check_validity parameter is added to BboxParams. Setting it to False gives a way to handle
bounding boxes extending beyond the image.
See motivation for it in #617.
Could anybody tell what is wrong with CI / Check code formatting with Black (3.8) here? I can not see any descriptive error message.
Is there anything blocking this PR from getting merged? This would be a very useful addition. Thanks!
Currently realization gives invalid results. We can see it by the test.
If we do not check validity, also we do not need check min_area and min_visibility.
In the test, we provide (-10, -20, 50, 150, 0) and get (0, 0, 25, 50, 0), but if we do not check validity we must get non clipped bbox: (-35.0, -45.0, 25.0, 125.0).
Otherwise I do not understand why we need this changes.
@Dipet, motivation is provided in #617.
Bound boxes resulting from augmentation are always clipped so that they fit inside resulting image. I don't propose to change it. The problem fixed here is that augmentations require initial boxes to be inside initial image. Let's try to describe motivation from #617 in some other words:
If you have image with boxes and for some reason you crop it before call to albumentations, you have to manually clip all boxes as well. The least problem is that it needs extra coding. But serious problem is that applying min_area and min_visibility criteria to such clipped boxes is erroneous because they do not represent whole objects.
If you need non-clipped rects as output (i.e. above result (-35.0, -45.0, 25.0, 125.0)), possibly it is usefull, it should be managed by quite another parameter (like it is done for keypoints, as far as I remembe) and represents another PR. As for me, I didn't need it until now. Remark: output boxes outside output image can result even from boxes inside an input image.
It is very strange annotation, where objects annotated outside field of view, because we do not know how objects looks outside field of view. For example some object may be occluded, but we do not know it, because we do not see it.
Please give me annotation example where it will be justified.
In my opinion - PR approach may lead to errors. Better to throw warning, that bbox is not clipped, and give the user a chance to fix the problem.
"Better to throw warning, that bbox is not clipped, and give the user a chance to fix the problem." - It is just what I am proposing! By default exception is rised, but if user needs it, he can fix it by setting parameter check_validity to false.
Here is quite a real usecase that forced me to make this PR.
I am working on human detection. A have annotated images, where human figures are large and hence often are cut by image borders. Such cut figures are annotated as humans as well.
When I train a network on cropped images, I need to ensure that bboxes annotated as humans are cropped not too much. It is just what min_visibility is designed for, right? Let's say, human must be visible at least at 50%. But if an initial box is already a part of human figure, cropped bbox with right visibility can represent inappropriate portion of a human figure (say, only hat). If I will use it for training it will spoil results.
I can estimate size of normal human figure at each place of image. So if annotation bbox is at the image border, I can evaluate the real bbox (including its part outside the image). And if I pass it to Albumentations, min_visibility filter will be applied correctly.
This real case is just one of possible similar situations. I've already proposed another one above: I can deal with only some part of image but annotation can be on on the whole image...
@Dipet Annotation of partly non-visible objects, be it truncation or occlusion, is helpful when you want to train your model to "guess" the object size. Let's say you want to do some 3D reconstruction, if you have a partial box, there is not much you can do. But if you have at least an estimate of the box size (bottom edge and height or width) then you can do at least some measurements and put a higher variance on these. I would very much like this feature, but rather without any flag. Basically just removing the assert and allowing boxes to be outside of the image. Are there any transformations where this could lead to problems?
A workaround I am using for now is converting the 4 corners of my bounding box to "keypoints" -> transform them -> then create the transformed bounding boxes from the new keypoints.
I would love to get this feature merged and wonder what is currently blocking this PR. If any input is required, I am happy to help. @Dipet
