Mask_RCNN
Mask_RCNN copied to clipboard
matterport
Cannot detect long thin objects
Hi, while working with the Mask RCNN I noticed that it cannot detect long thin-ish objects (e.g. a pipe or a wire). It is possible that I do not have enough samples (small and imbalanced dataset), but I was wondering if anyone has had a similar issue and how you solved it (or suggestions to check).
@q1nwu, how to change RPN_ANCHOR_RATIOS, the default value is [0.5, 1, 2]; if set it as [0.5, 1, 2, 4, 8], that will be error. It seems to only need three values for RPN_ANCHOR_RATIOS like [2, 4, 8]?
Adding more ratios may not work either if the long thin object is not vertical or horizontal (e.g., 45 degrees oblique wire will have a bbox of aspect ratio 1:1). If your dataset is filled with such cases, I would argue it is hard for Mask RCNN to learn as the mask to bbox ratio is too low.
@t-chenx I guess you use coco weights. I think you should use ImageNet weights.
@patrick-12sigma Thank you for your thought. It does make sense, so I will investigate this further.
I am having similar issues. Would appreciate if anybody has some insights. @patrick-12sigma Is low mask to bbox ratio a un-solvable problem in mask-rcnn?
@YubinXie Actually I began to think it is a solvable problem. If you look at the example results in Mask RCNN paper, there are many long thin objects posed at almost 45 degrees. I agree that in that case you should perhaps use more aspect ratios. However you may need to exclude more layers when loading COCO weight to avoid layer dimension mismatch.
I did achieve better segmentations for long thin objects by having more samples in the dataset. If I remember correctly, having more aspect ratios did not improve the detections.
@g2-bernotas Hi, I'm also interested in the problem. By saying more samples in the dataset, do you mean having more training samples? Is there any other config do you recommend to change in order to improve the performance? Thanks!
@g2-bernotas how many samples did you end up using? I've got more than 1000 and it still doesn't really work. I get dots across the object instead of a nice clean segmentation. https://github.com/matterport/Mask_RCNN/issues/853
I am wondering what is the configuration for you guys to detect long thin objects? I was using the config of detecting shapes in here [(https://github.com/matterport/Mask_RCNN/tree/master/samples/shapes)]. It worked fine. But when I tried to improve it and used coco's config, the accuracy dramatically decreased.
The long thin object seems to need a special training configuration.
Got similar problem. I tried the coco model on animal detection, 100 training samples. Without the tail, it worked pretty good. With tail added in mask, the performance dramatically degradated.
the values of RPN_ANCHOR_RATIOS are width/height or height/width? why the mark said 0.5 is a wide anchor ?
@banafsh89 I don't have a particular solution to this problem. It seems that it was particularly difficult for very thin objects despite having different sizes of RPN_ANCHOR_RATIOS. We used Mask RCNN for two purposes - separate individual plant leaves (stem was the problem at first, but got better without any modifications to the training procedure) and hanging wires (this could not be solved to our required accuracy). What objects are you interested in separating?
@g2-bernotas My objects are actually opposite of yours. I have some over density colors which can become large or small. I have the same problem as you have but not for small long object, for very big objects. the network will see the big object as several small objects. I thought that your solution can shed lit on mine too.
the values of RPN_ANCHOR_RATIOS are width/height or height/width? why the mark said 0.5 is a wide anchor ?
@lunasdejavu i had same question at first, then after checking this notebook https://github.com/matterport/Mask_RCNN/blob/v2.1/samples/balloon/inspect_balloon_data.ipynb, i can confirm that it is width/height, so the comments for 0.5 should be tall anchor.
to be honest, its 2024, and i have been trying to tackle a similar issue, in my case am detecting potted grasses (mind that grasses have spread yet very thin leaves), these grasses might, e.g., span an enclosing box (BBox) of size 400X390px, but the actual plant material (i.e., pixels belonging to the plant leaves) counts to less than 10% of the bbox (mask to box ratio). in some cases.
not directly relevant to this, but also the fact that coco doesn't natively handle nested polygons, I figured the mask to box ratio gets much lower because of the poly to mask conversion routines in coco where inner holes get mis-labelled frequently. e.g., in such cases nested holes are labelled plants, where in fact they are background pixels... this increases the imbalance in during modelling and leads to a terrible performance..
any updates on this would be very much appreciated.
