mmpose icon indicating copy to clipboard operation
mmpose copied to clipboard

[Feature] Support ViTPose

Open LareinaM opened this issue 2 years ago • 1 comments

Motivation

Add implementation of ViTPose on MMPose 1.0

Modification

  • Add six config files
  • Add layer decay optimizer
  • In HeatmapHead, add parameter upsample, default to zero (no effect on previous codes), resize in BaseHead

BC-breaking (Optional)

Use cases (Optional)

Checklist

**Before

  • [x] I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
  • [x] Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
  • [x] Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
  • [x] New functionalities are covered by complete unit tests. If not, please add more unit tests to ensure correctness.
  • [ ] The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

  • [ ] CLA has been signed and all committers have signed the CLA in this PR.

LareinaM avatar Dec 12 '22 03:12 LareinaM

Have you tried training a base ViT model to check the accuracy?

jin-s13 avatar Dec 12 '22 10:12 jin-s13

Result of the current implementation

With classic decoder

Arch Input Size AP AR
ViTPose-S 256x192 0.739 0.792
ViTPose-B 256x192 0.757 0.810
ViTPose-L 256x192 0.782 0.834
ViTPose-H 256x192 0.788 0.839

With simple decoder

Arch Input Size AP AR
ViTPose-S 256x192 0.736 0.790
ViTPose-B 256x192 0.756 0.809
ViTPose-L 256x192 0.781 0.833
ViTPose-H 256x192 0.789 0.839

Result of original ViTPose implementation

With classic decoder

Model Input Size AP AR
ViTPose-S 256x192 0.738 0.792
ViTPose-B 256x192 0.758 0.811
ViTPose-L 256x192 0.783 0.835
ViTPose-H 256x192 0.791 0.841

With simple decoder

Model Input Size AP AR
ViTPose-S 256x192 0.735 0.789
ViTPose-B 256x192 0.755 0.809
ViTPose-L 256x192 0.782 0.834
ViTPose-H 256x192 0.789 0.840

LareinaM avatar Feb 07 '23 12:02 LareinaM

Codecov Report

Patch coverage: 6.34% and project coverage change: -0.46 :warning:

Comparison is base (d341f11) 82.22% compared to head (e22d6c8) 81.77%.

:exclamation: Current head e22d6c8 differs from pull request most recent head 23b7838. Consider uploading reports for the commit 23b7838 to get more accurate results

Additional details and impacted files
@@             Coverage Diff             @@
##           dev-1.x    #1876      +/-   ##
===========================================
- Coverage    82.22%   81.77%   -0.46%     
===========================================
  Files          225      227       +2     
  Lines        13375    13438      +63     
  Branches      2269     2285      +16     
===========================================
- Hits         10998    10989       -9     
- Misses        1864     1933      +69     
- Partials       513      516       +3     
Flag Coverage Δ
unittests 81.77% <6.34%> (-0.46%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
mmpose/engine/optim_wrappers/__init__.py 0.00% <0.00%> (ø)
...engine/optim_wrappers/layer_decay_optim_wrapper.py 0.00% <0.00%> (ø)
mmpose/models/heads/heatmap_heads/heatmap_head.py 82.19% <23.07%> (-5.78%) :arrow_down:
mmpose/models/heads/base_head.py 81.81% <33.33%> (-2.31%) :arrow_down:

... and 2 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

codecov[bot] avatar Feb 15 '23 13:02 codecov[bot]