[Algorithm] Support PSC

[yuyi1005]

Motivation

Support PSC, phase-shifting coder and dual-frequency phase-shifting coder.

Modification

1. An angle coder named PSCCoder is added to mmrotate/models/task_modules/coders/angle_coder.py.
2. Name encoded_size is unified to encode_size to comform with other codes (both encode_size and encoded_size are used in codes, which I believe should be further unified, please tell me which one is prefered).
3. A slight amendment is made to mmrotate/models/dense_heads/angle_branch_retina_head.py to support both CSL and PSC (the original verion use angle_coder['omega'] to calculate the encoded_size, which can only work with CSL).
4. The configuration files for running PSC are added to configs/psc.

Performance are as follows:

DOTA:

Method	mAP50	mAP75	mAP50:95
rotated_fcos_psc_dual_r50_fpn_1x_dota_rr_le90	72.90	39.80	41.51
rotated_retinanet_obb_psc_dual_r50_fpn_fp16_1x_dota_le90	71.09	41.17	41.25

HRSC:

Method	mAP50	mAP75	mAP50:95
rotated_fcos_psc_r50_fpn_6x_hrsc_le90	90.06	78.56	67.57
rotated_retinanet_obb_psc_r50_fpn_6x_hrsc_le90	85.65	61.30	54.14

1. Above results are obtained on mmrotate 0.3.2.
2. This pull request has been tested on dev-1.x, the provided two config files can run without error (both based on retinanet).
3. But the accuracy is mAP50=66.5 (DOTA, val) and mAP50=82.0 (HRSC), lower than that on mmrotate 0.3.2 (the basic rotated_retinanet and rotated_retinanet+csl also show lower mAP on 1.x)
4. rotated_fcos+psc failed in training (so does rotated_fcos+csl, meaning that this bug is not induced by PSC).

[liuyanyi]

Hi, thanks for your great work. Can you provided the cfg and error message of fcos, I'll have a check.

[yuyi1005]

I copied the rotated-fcos-hbox-le90_r50_fpn_csl-gaussian_1x_dota.py, deleted 'use_hbbox_loss=True', and then the error occured. With use_hbbox_loss=True, both CSL and PSC can work now. I think I was misled by the name "rotated-fcos-hbox-le90_r50_fpn_csl-gaussian_1x_dota", the name seems to mean that this config will output horizontal boxes.

[liuyanyi]

I copied the rotated-fcos-hbox-le90_r50_fpn_csl-gaussian_1x_dota.py, deleted 'use_hbbox_loss=True', and then the error occured. With use_hbbox_loss=True, both CSL and PSC can work now. I think I was misled by the name "rotated-fcos-hbox-le90_r50_fpn_csl-gaussian_1x_dota", the name seems to mean that this config will output horizontal boxes.

In 1.x fcos, the parameters are a bit different from 0.x, you can see the diff in #543 . 'hbb' in cfg name means 'use_hbbox_loss=True'.

When set use_hbbox_loss=True, the bbox loss will apply on hbox (x,y,w,h) and angle loss is required , so 'mmdet.IoULoss' is used.

It seems PSC's decoding process is differentiable, you can try PSC with Rotated IoU Loss too.

[yuyi1005]

Thanks, I have just finished the training on HRSC, the mAP50 of FCOS+PSC is 90.1

[Leael]

hello, can I ask how to infer this? I tried to infer this one but I got this error KeyError: "class RetinaNet in mmdet/models/detectors/retinanet.py: 'AngleBranchRetinaHead is not in the model registry. Please check whether the value of AngleBranchRetinaHead is correct or it was registered as expected. More details can be found at https://mmengine.readthedocs.io/en/latest/tutorials/config.html#import-custom-python-modules'"

[yuyi1005]

hello, can I ask how to infer this? I tried to infer this one but I got this error KeyError: "class RetinaNet in mmdet/models/detectors/retinanet.py: 'AngleBranchRetinaHead is not in the model registry. Please check whether the value of AngleBranchRetinaHead is correct or it was registered as expected. More details can be found at https://mmengine.readthedocs.io/en/latest/tutorials/config.html#import-custom-python-modules'"

Can you provide the command to reproduce this error? AngleBranchRetinaHead is registered by default, instead of by me. Can you try CSL which is also based on AngleBranchRetinaHead?

[Leael]

hello, can I ask how to infer this? I tried to infer this one but I got this error KeyError: "class RetinaNet in mmdet/models/detectors/retinanet.py: 'AngleBranchRetinaHead is not in the model registry. Please check whether the value of AngleBranchRetinaHead is correct or it was registered as expected. More details can be found at https://mmengine.readthedocs.io/en/latest/tutorials/config.html#import-custom-python-modules'"

Can you provide the command to reproduce this error? AngleBranchRetinaHead is registered by default, instead of by me. Can you try CSL which is also based on AngleBranchRetinaHead?

this is the command I used import cv2 import mmcv from mmcv.transforms import Compose from mmengine.utils import track_iter_progress from mmdet.registry import VISUALIZERS from mmdet.utils import register_all_modules from mmdet.apis import init_detector, inference_detector

register_all_modules()

config_file = '/mmrotate/configs/psc/rotated-retinanet-rbox-le90_r50_fpn_psc-dual_amp-1x_dota.py' checkpoint_file = '/work_dir/epoch_12.pth'

model = init_detector(config_file, checkpoint_file, device='cuda:0')

visualizer = VISUALIZERS.build(model.cfg.visualizer)

visualizer.dataset_meta = model.dataset_meta

[yuyi1005]

@Leael from mmdet.utils import register_all_modules as register_all_modules_mmdet from mmrotate.utils import register_all_modules

register_all_modules_mmdet() register_all_modules()

[Leael]

@Leael from mmdet.utils import register_all_modules as register_all_modules_mmdet from mmrotate.utils import register_all_modules

register_all_modules_mmdet() register_all_modules()

Thanks! It worked

but when I add this code to show the result. visualizer.add_datasample( 'result', img, data_sample=result, draw_gt=False, show=True)

I got this error AttributeError: 'Tensor' object has no attribute 'convert_to'

[yuyi1005]

@Leael That is a bug caused by the different definition of bbox between visualization and inference, which is supposed to be fixed by mainteners. I can give you a temporary solution: mmrotate\visualization\local_visualizer.py, line 72, add these two lines: from mmrotate.structures.bbox import RotatedBoxes bboxes = RotatedBoxes(bboxes, dtype=torch.float32)

[yuyi1005]

@zytx121 Hi, the ci is stuck at "First-time contributors need a maintainer to approve running workflows.". Could you find a authorized maintainer to approve it for me

[zytx121]

@zytx121 Hi, the ci is stuck at "First-time contributors need a maintainer to approve running workflows.". Could you find a authorized maintainer to approve it for me

Thanks for your contribution! I will promote this PR as soon as possible.

Your CI has passed . Please don't care the error about build_windows. We have fix it in https://github.com/open-mmlab/mmrotate/pull/621

[codecov[bot]]

Codecov Report

:exclamation: No coverage uploaded for pull request base (dev-1.x@766185e). Click here to learn what that means. Patch has no changes to coverable lines.

Additional details and impacted files

@@            Coverage Diff             @@
##             dev-1.x     #617   +/-   ##
==========================================
  Coverage           ?   79.76%           
==========================================
  Files              ?       95           
  Lines              ?     6448           
  Branches           ?      912           
==========================================
  Hits               ?     5143           
  Misses             ?     1008           
  Partials           ?      297           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[yangxue0827]

Check the table please.

[OpenMMLab-Assistant005]

Dear yuyi1005, First of all, we want to express our gratitude for your significant PR in the MMRotate project. Your contribution is highly appreciated, and we are grateful for your efforts in helping improve this open-source project during your personal time. We believe that many developers will benefit from your PR. If you are Chinese or have WeChat，welcome to join our community on WeChat. You can add our assistant ：openmmlabwx. Please add "mmsig + Github ID" as a remark when adding friends：） We would also like to invite you to join our Special Interest Group (SIG) private channel on Discord, where you can share your experiences, ideas, and build connections with like-minded peers. To join the SIG channel, simply message moderator— OpenMMLab on Discord or briefly share your open-source contributions in the #introductions channel and we will assist you. We look forward to seeing you there! Join us ：https://discord.gg/raweFPmdzG Thank you again for your contribution❤ Best regards！ @yuyi1005