Fix swin backbone absolute pos_embed

[normster]

trafficstars

Motivation

The Swin transformer backbone is currently unable to process input images differing in size from the pretraining resolution when absolute positional embedding is enabled, due to lack of dynamic pos embed resizing in the forward pass.

Modification

When absolute positional embedding is enabled, the backbone now checks the input shape against the pos embed shape and performs a bicubic interpolation to the correct shape if necessary. This is based on the original Mask2former code: https://github.com/facebookresearch/Mask2Former/blob/main/mask2former/modeling/backbone/swin.py#L656.

The pos embed parameter's shape has also been changed from [B * (H*W) * D] -> [B * D * H * W] to bring the model into consistency with the checkpoint loading code, which expects the new shape.

BC-breaking (Optional)

This fix should not break backwards compatibility. No existing configs using the Swin backbone have absolute positional embedding enabled, or load from

Use cases (Optional)

Checklist

1. Pre-commit or other linting tools are used to fix the potential lint issues.
2. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
3. If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMCls.
4. The documentation has been modified accordingly, like docstring or example tutorials.

[CLAassistant]

All committers have signed the CLA.

[codecov[bot]]

Codecov Report

Merging #8127 (605a55b) into dev (ca11860) will increase coverage by 0.00%. The diff coverage is 60.00%.

@@           Coverage Diff           @@
##              dev    #8127   +/-   ##
=======================================
  Coverage   64.17%   64.17%           
=======================================
  Files         361      361           
  Lines       29525    29529    +4     
  Branches     5019     5020    +1     
=======================================
+ Hits        18947    18950    +3     
- Misses       9575     9576    +1     
  Partials     1003     1003           

Flag	Coverage Δ
unittests	64.15% <60.00%> (+<0.01%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmdet/models/backbones/swin.py	81.01% <60.00%> (-0.49%)	:arrow_down:
mmdet/datasets/pipelines/loading.py	57.99% <0.00%> (+0.15%)	:arrow_up:
mmdet/utils/misc.py	64.10% <0.00%> (+2.56%)	:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[chhluo]

please add a unit test in https://github.com/open-mmlab/mmdetection/blob/master/tests/test_models/test_backbones/test_swin.py for the modification

[ZwwWayne]

This PR needs to be migrated to dev-3.x

[OpenMMLab-Assistant-004]

Hi @normster ！First of all, we want to express our gratitude for your significant PR in the mmdetection project. Your contribution is highly appreciated, and we are grateful for your efforts in helping improve this open-source project during your personal time. We believe that many developers will benefit from your PR.

We would also like to invite you to join our Special Interest Group (SIG) private channel on Discord, where you can share your experiences, ideas, and build connections with like-minded peers. To join the SIG channel, simply message moderator— OpenMMLab on Discord or briefly share your open-source contributions in the #introductions channel and we will assist you. Look forward to seeing you there! Join us ：https://discord.gg/raweFPmdzG

If you have WeChat，welcome to join our community on WeChat. You can add our assistant ：openmmlabwx. Please add "mmsig + Github ID" as a remark when adding friends：） Thank you again for your contribution❤