mmpose
mmpose copied to clipboard
[Feature] Support ViTPose
Motivation
Add implementation of ViTPose on MMPose 1.0
Modification
- Add six config files
- Add layer decay optimizer
- In
HeatmapHead
, add parameterupsample
, default to zero (no effect on previous codes),resize
inBaseHead
BC-breaking (Optional)
Use cases (Optional)
Checklist
**Before
- [x] I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
- [x] Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
- [x] Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
- [x] New functionalities are covered by complete unit tests. If not, please add more unit tests to ensure correctness.
- [ ] The documentation has been modified accordingly, including docstring or example tutorials.
After PR:
- [ ] CLA has been signed and all committers have signed the CLA in this PR.
Have you tried training a base ViT model to check the accuracy?
Result of the current implementation
With classic decoder
Arch | Input Size | AP | AR |
---|---|---|---|
ViTPose-S | 256x192 | 0.739 | 0.792 |
ViTPose-B | 256x192 | 0.757 | 0.810 |
ViTPose-L | 256x192 | 0.782 | 0.834 |
ViTPose-H | 256x192 | 0.788 | 0.839 |
With simple decoder
Arch | Input Size | AP | AR |
---|---|---|---|
ViTPose-S | 256x192 | 0.736 | 0.790 |
ViTPose-B | 256x192 | 0.756 | 0.809 |
ViTPose-L | 256x192 | 0.781 | 0.833 |
ViTPose-H | 256x192 | 0.789 | 0.839 |
Result of original ViTPose implementation
With classic decoder
Model | Input Size | AP | AR |
---|---|---|---|
ViTPose-S | 256x192 | 0.738 | 0.792 |
ViTPose-B | 256x192 | 0.758 | 0.811 |
ViTPose-L | 256x192 | 0.783 | 0.835 |
ViTPose-H | 256x192 | 0.791 | 0.841 |
With simple decoder
Model | Input Size | AP | AR |
---|---|---|---|
ViTPose-S | 256x192 | 0.735 | 0.789 |
ViTPose-B | 256x192 | 0.755 | 0.809 |
ViTPose-L | 256x192 | 0.782 | 0.834 |
ViTPose-H | 256x192 | 0.789 | 0.840 |
Codecov Report
Patch coverage: 6.34
% and project coverage change: -0.46
:warning:
Comparison is base (
d341f11
) 82.22% compared to head (e22d6c8
) 81.77%.
:exclamation: Current head e22d6c8 differs from pull request most recent head 23b7838. Consider uploading reports for the commit 23b7838 to get more accurate results
Additional details and impacted files
@@ Coverage Diff @@
## dev-1.x #1876 +/- ##
===========================================
- Coverage 82.22% 81.77% -0.46%
===========================================
Files 225 227 +2
Lines 13375 13438 +63
Branches 2269 2285 +16
===========================================
- Hits 10998 10989 -9
- Misses 1864 1933 +69
- Partials 513 516 +3
Flag | Coverage Δ | |
---|---|---|
unittests | 81.77% <6.34%> (-0.46%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
Impacted Files | Coverage Δ | |
---|---|---|
mmpose/engine/optim_wrappers/__init__.py | 0.00% <0.00%> (ø) |
|
...engine/optim_wrappers/layer_decay_optim_wrapper.py | 0.00% <0.00%> (ø) |
|
mmpose/models/heads/heatmap_heads/heatmap_head.py | 82.19% <23.07%> (-5.78%) |
:arrow_down: |
mmpose/models/heads/base_head.py | 81.81% <33.33%> (-2.31%) |
:arrow_down: |
... and 2 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.