Using Mask Scoring RCNN in mmdet 3.0+

[willcray]

Hello, I was happily using mmdet 2.28.0 and the Mask IoU / Mask Quality scores outputted by the mask_iou_head in the the Mask Scoring RCNN model. I recently upgraded to mmdet 3.0, and now the output of the inference_detector doesn't include Mask IoU scores. The output is a DetDataSample whose pred_instances only contain bbox, scores (for the bounding boxes), labels, and masks. There is no mask scores / mask quality scores / mask iou scores field.

How do I use models that output mask iou scores within the new context of a DetDataSample?

[matthost]

We were using Mask Scoring RCNN in 2.x. Upgrading now. Noticed there are no longer configs for using mask scoring rcnn. No longer supported?

[willcray]

We were using Mask Scoring RCNN in 2.x. Upgrading now. Noticed there are no longer configs for using mask scoring rcnn. No longer supported?

There are configs in mmdet 3.0+ here

[matthost]

Did you end up getting it working in 3.x ?

[willcray]

Did you end up getting it working in 3.x ?

I can use a MaskScoringRCNN model, but I can't access its mask scores.

[matthost]

In training I am seeing an error:

2024-02-08T20:45:39.643Z | File "/opt/conda/lib/python3.10/site-packages/mmdet/models/roi_heads/standard_roi_head.py", line 140, in loss
-- | --
  | 2024-02-08T20:45:39.643Z | mask_results = self.mask_loss(x, sampling_results,
  | 2024-02-08T20:45:39.643Z | File "/opt/conda/lib/python3.10/site-packages/mmdet/models/roi_heads/mask_scoring_roi_head.py", line 141, in mask_loss
  | 2024-02-08T20:45:39.643Z | loss_mask_iou = self.mask_iou_head.loss_and_target(
  | 2024-02-08T20:45:39.643Z | File "/opt/conda/lib/python3.10/site-packages/mmdet/models/roi_heads/mask_heads/maskiou_head.py", line 142, in loss_and_target
  | 2024-02-08T20:45:39.643Z | mask_iou_targets = self.get_targets(
  | 2024-02-08T20:45:39.643Z | File "/opt/conda/lib/python3.10/site-packages/mmdet/models/roi_heads/mask_heads/maskiou_head.py", line 193, in get_targets
  | 2024-02-08T20:45:39.643Z | area_ratios = torch.cat(list(area_ratios))
  | 2024-02-08T20:45:39.643Z | File "/opt/conda/lib/python3.10/site-packages/mmdet/models/roi_heads/mask_heads/maskiou_head.py", line 240, in _get_area_ratio
  | 2024-02-08T20:45:39.644Z | ratio = gt_mask_in_proposal.areas[0] / (
  | 2024-02-08T20:45:39.644Z | IndexError: index 0 is out of bounds for axis 0 with size 0