[Feature] Multi task head

[piercus]

Motivation

I've read the #628 issue and implemented a proposal for MultiTaskClsHead in order to mix multiple classification head to create a multi task classification tool.

Modification

Please briefly describe what modification is made in this PR.

• Created a unit test for MultiTaskClsHead
• the implementation of MultiTaskClsHead

NB : Current implementation is made to be generic and the different head can output different shapes (it will give the ability to mix head from mmcls with heads from mmpose or other projects). The drawback of this is that the output of the simple_test method cannot be a tensor, i choose to deactivate the post_process=False mode for the simple_test method. Any suggestion would be welcome

BC-breaking (Optional)

No

Use cases (Optional)

The basic use case here it to classify multiple classes using the same neural network. In order to make it work, it is needed to implement a compatible dataset.

I have made the following :

• Starting from the ImageNet dataset
• in load_annotations, info['gt_label'] is a tuple containing one gt_label per head
• evaluate method need to be reworked

Checklist

Before PR:

• [x] Pre-commit or other linting tools are used to fix the potential lint issues.
• [x] Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
• [x] The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
• [ ] The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

• [ ] If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects, like MMDet or MMSeg.
• [x] CLA has been signed and all committers have signed the CLA in this PR.

[CLAassistant]

All committers have signed the CLA.

[mzr1996]

Sounds like a good feature. We will review it recently. In addition, MMCLS is now developed based on the dev branch, and I have switched this branch's base to dev. More details can be found in Develop based on MMClassification

[codecov[bot]]

Codecov Report

Merging #675 (df68f03) into dev (59292b3) will increase coverage by 0.04%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##              dev     #675      +/-   ##
==========================================
+ Coverage   87.02%   87.07%   +0.04%     
==========================================
  Files         130      131       +1     
  Lines        8538     8571      +33     
  Branches     1468     1476       +8     
==========================================
+ Hits         7430     7463      +33     
  Misses        888      888              
  Partials      220      220              

Flag	Coverage Δ
unittests	86.99% <100.00%> (+0.05%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmcls/models/heads/__init__.py	100.00% <100.00%> (ø)
mmcls/models/heads/multi_task_head.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update bce95b9...df68f03. Read the comment docs.

[Ezra-Yu]

Thank you for your contribution, please follow this pre-commit guide to fix the lint.

[piercus]

@mzr1996 @Ezra-Yu Thanks for your support !

With the switch of the base, i feel this PR was carrying a lot of non-multi_task_head-related changes (maybe some change to merge from master to dev)

I have cleaned up everything from the dev branch, cherry-picking my changes, so the PR contains only relevant changes I also ran pre-commit in order to fix the linting.

Now the PR should be more clean and easier to review.

[ck37]

Thanks for the work on this! I need to combine an image classification head with an mmdet object detection model as well as mmocr, so would really appreciate this being possible in a single combined model.

[YilanWang]

I think you need to give some examples in Class commit, thanks for your work! It is very helpful! @piercus

[mzr1996]

Hello, I have modified some APIs to support the entire multi-task design. More details can be found in #808 In the design, we use a dict instead of a tuple to save the ground-truth labels passed to MultiTaskClsHead. And I added the support of post_process=False in the simple_test.

[Ezra-Yu]

this feature has been supported in the new main branch, so close it here.