mmdetection

mmdetection copied to clipboard

We keep this issue open to collect feature requests from users and hear your voice. Our monthly release plan will be updated in different issues.

In this issue, you can either:

1. Suggest a new feature by leaving a comment.
2. Vote for a feature request with :+1: or be against with :-1:. (Remember that developers are busy and cannot respond to all feature requests, so vote for your most favorable one!)
3. Tell us that you would like to help implement one of the features in the list or review the PRs. (This is the greatest things to hear about!)

We also released our TODO list on the project page. Most of the TODO items are described in their corresponding issues (those labeled by Dev-RD) with detailed requirement documentation. Feel free to leave a message in the issue of any item and create a PR if you are interested in any of the item.

I would be interested in adding support for EfficientNet (backbone, #764) and EfficientDet (detector, #1827) architectures along with trained models.

I will also be willing to contribute this feature myself.

It would be interesting to add batch inference support. I've seen a couple of issues related with this (#2703 #1833 #1659) and some details in source code and it looks like it has been left as future work (see https://github.com/open-mmlab/mmdetection/blob/master/mmdet/models/detectors/base.py#L118) so it could be a good moment.

I'd be glad to help with this too.

How about adding the support for yolov4?

Hi authors Can you add DOTA dataset support? ex:more bbox parameters! https://captain-whu.github.io/DOTA/dataset.html

Thank you so much!!

Any plans about knowledge distillation for object detection, for example, fitnet (https://github.com/TuSimple/simpledet/blob/master/models/KD/README.md), so that users can customize their distillation loss functions based on their requirements.

I would be interested in adding support for EfficientNet (backbone, #764) and EfficientDet (detector, #1827) architectures along with trained models.

I will also be willing to contribute this feature myself.

Looking forward to it. Is there expected time for EfficientNet and EfficientDet? We need to have a plan whether to release it in V2.2 (June 30) or V2.3 (July 31).

It would be interesting to add batch inference support. I've seen a couple of issues related with this (#2703 #1833 #1659) and some details in source code and it looks like it has been left as future work (see https://github.com/open-mmlab/mmdetection/blob/master/mmdet/models/detectors/base.py#L118) so it could be a good moment.

I'd be glad to help with this too.

Thanks! We have already created a PR (#1833) to support batch inference a few months ago, and will continue working on it to make it compatible with V2.0. You are still welcome to contribute any other new features.

How about adding the support for yolov4?

If there is no community contributors helping with that, we will firstly add YOLO v3 in V2.3 (July 31).

Hi authors Can you add DOTA dataset support? ex:more bbox parameters! https://captain-whu.github.io/DOTA/dataset.html

Think you so much!!

For now we do not have plan for that, but there have been mmdet-based project doing that. You might be interested in https://github.com/dingjiansw101/AerialDetection.

I would be interested in adding support for EfficientNet (backbone, #764) and EfficientDet (detector, #1827) architectures along with trained models. I will also be willing to contribute this feature myself.

Looking forward to it. Is there expected time for EfficientNet and EfficientDet? We need to have a plan whether to release it in V2.2 (June 30) or V2.3 (July 31).

I think that probably EfficientNet for version v2.2 and EfficientDet for v2.3

Hi ! authors Any plan about AC-FPN ? (https://arxiv.org/pdf/2005.11475.pdf

Could you add some help about how to actually build modules out of these existing ones?

Hi @SuzaKrish , Tutorials have been already added here. Is there anything unclear or not sufficient?

Thanks! So in the tutorials, you have mentioned how the base is supposed to be formed right? how do we then call it finally to run the model? Also, while going over the config files, I had a doubt as to where to find these modules in the main repo page. Could you probably include a part in the readme describing which folders contain which components(like head, model structure, base etc.)

Hi @SuzaKrish , This documentation might help you.

How about supporting coco-like evaluation for custom datasets without coco json annotation file? For instance, by converting the custom dataset to coco format internally during evaluation like https://github.com/facebookresearch/detectron2/blob/master/detectron2/evaluation/coco_evaluation.py#L72.

documentation

Thank you! This is helpful! :) @ZwwWayne

Is there any plan for supporting some light-weight networks like BiSeNet, DFANet?

Any plans to support light-weight backbone such as mobilenetV2, mobilenetV3 ?

How about adding the support for yolov4?

If there is no community contributors helping with that, we will firstly add YOLO v3 in V2.3 (July 31).

Hello, @WenqiangX and I are glad to help implement the YOLO v3, and perhaps v4 as well, if we feel good. We are from MVIG, SJTU and have spent quite a little time studying all kinds of YOLOv3 implementation, especially the one from gluon-cv, which is probably one of the best re-implementation of YOLOv3.

Basically, we plan to continue the work of #1695 . From a big picture, we plan to do the following:

1. Refactor the backbone with your new ConvModule (as mentioned in https://github.com/open-mmlab/mmdetection/pull/1695#issuecomment-598038394)
2. Solve the problem of not being able to use Distribute Training
3. Introduce some great ideas from gluon-cv

If you are glad with us, can you manage your time to check if there is any major defect in #1695, so that we can save some review time in the future?

BTW, I don't know if the license from Western Digital company will be a big issue.

Any plan on adding vovnet backbone, such as https://github.com/aim-uofa/AdelaiDet/tree/master/configs/FCOS-Detection/vovnet?

How about adding the support for yolov4?

If there is no community contributors helping with that, we will firstly add YOLO v3 in V2.3 (July 31).

Hello, @WenqiangX and I are glad to help implement the YOLO v3, and perhaps v4 as well, if we feel good. We are from MVIG, SJTU and have spent quite a little time studying all kinds of YOLOv3 implementation, especially the one from gluon-cv, which is probably one of the best re-implementation of YOLOv3.

Basically, we plan to continue the work of #1695 . From a big picture, we plan to do the following:

1. Refactor the backbone with your new ConvModule (as mentioned in #1695 (comment))
2. Solve the problem of not being able to use Distribute Training
3. Introduce some great ideas from gluon-cv

If you are glad with us, can you manage your time to check if there is any major defect in #1695, so that we can save some review time in the future?

BTW, I don't know if the license from Western Digital company will be a big issue.

Thanks and glad to know you are willing to help! We can have further discussion in that PR and may expect YOLOv3 in V2.3. The copyright is ok as if it is licensed under Apache-2.0.

Can we add SpineNet (CVPR2020)? The code is provided by @lucifer443 https://github.com/lucifer443/SpineNet-Pytorch As said in the repo, I don't have enough servers to train it, but willing to adjust the code and pull request.

I only trained with protocol B. Training SpineNet takes lots of time, for example I took 7 days to train SpineNet-49 with 8 TITAN V gpus.

Can we add SpineNet (CVPR2020)? The code is provided by @lucifer443 https://github.com/lucifer443/SpineNet-Pytorch As said in the repo, I don't have enough servers to train it, but willing to adjust the code and pull request.

I only trained with protocol B. Training SpineNet takes lots of time, for example I took 7 days to train SpineNet-49 with 8 TITAN V gpus.

PRs are welcomed.

any plan for ONNX support? e.g. two stage faster-rcnn thanks.

"Bag of Freebies for Training Object Detection Neural Networks"

Describe the feature Is it possible to support the training improvements described in the following paper: "Bag of Freebies for Training Object Detection Neural Networks" Zhi Zhang, Tong He, Hang Zhang, Zhongyue Zhang, Junyuan Xie, Mu Li https://arxiv.org/pdf/1902.04103.pdf

Motivation It seems the "Bag of Freebies" provide significant accuracy improvement (+5%) in a single shot detector such as YOLOv3. It improves Faster-RCNN by upto 1.7%. It is likely that these features will improve other detectors as well.

Related resources https://medium.com/apache-mxnet/gluoncv-0-3-a-new-horizon-564326364e16 https://gluon-cv.mxnet.io/model_zoo/detection.html

Additional context I think implementing these features would give accuracy lift to several object detectors in mmdetection.

Note: Added here as per the suggetion in https://github.com/open-mmlab/mmdetection/issues/3124

Thanks,

"Objects as Points (CenterNet)" is a popular, high accuracy Object Detector.

"Objects as Points" Xingyi Zhou, Dequan Wang, Philipp Krähenbühl https://arxiv.org/abs/1904.07850 Source code: https://github.com/xingyizhou/CenterNet (This detector is referred to as CenterNet in the paper. But there is another detector that calls itself CenterNet - so I am using both names to avoid confusion)

In CVPR2020, the second place solution in the 2D Object detection track of the Waymo open challenges used this detector as one component: https://waymo.com/open/challenges/ https://waymo.com/open/challenges/2d-detection/ "2nd Place Solution for Waymo Open Dataset Challenge - 2D Object Detection" Sijia Chen∗ Yu Wang∗ Li Huang Runzhou Ge Yihan Hu Zhuangzhuang Ding Jie Liao, Horizon Robotics Inc. https://arxiv.org/pdf/2006.15507.pdf

It is also worth noting that this CenterNet inspired AFDet which in-turn was the basis for in the 1st place Solution for 3D Object Detection: "1st Place Solution for Waymo Open Dataset Challenge - 3D Detection and Domain Adaptation" Zhuangzhuang Ding∗ Yihan Hu∗ Runzhou Ge∗ Li Huang Sijia Chen Yu Wang Jie Liao Horizon Robotics https://arxiv.org/pdf/2006.15505.pdf "AFDet: Anchor Free One Stage 3D Object Detection" Runzhou Ge∗ Zhuangzhuang Ding∗ Yihan Hu∗ Yu Wang Sijia Chen Li Huang Yuan Li Horizon Robotics https://arxiv.org/pdf/2006.12671.pdf

So, having this "Objects as Points (CenterNet)" Detector in mmdetection is highly desirable. Kindly add it if possible.

"Objects as Points (CenterNet)" is a popular, high accuracy Object Detector.

Objects as Points Xingyi Zhou, Dequan Wang, Philipp Krähenbühl https://arxiv.org/abs/1904.07850 Source code: https://github.com/xingyizhou/CenterNet (This detector is referred to as CenterNet in the paper. But there is another detector that calls itself CenterNet - so I am using both names to avoid confusion)

In CVPR2020, the second place solution in the 2D Object detection track of the Waymo open challenges used this detector as one component: https://waymo.com/open/challenges/ https://waymo.com/open/challenges/2d-detection/

2nd Place Solution for Waymo Open Dataset Challenge - 2D Object Detection Sijia Chen∗ Yu Wang∗ Li Huang Runzhou Ge Yihan Hu Zhuangzhuang Ding Jie Liao, Horizon Robotics Inc. https://arxiv.org/pdf/2006.15507.pdf

Having this "Objects as Points (CenterNet)" Detector in mmdetection is highly desirabale. Kindly add it if possible.

hey @mathmanu maybe this PR will help you. But it is not offically supported and only work with mmdet v1.x. Would you mind merging it with newer verison ? @hellock @ZwwWayne I strongly believe our community want this one detector in the model zoo.

Hi @mathmanu , @mathmanu , Thanks for your kind suggestion. Due to limited developers and resources, we are not going to implement those methods in the near future, PRs are welcome.

Hi @hellock @xvjiarui ,

The main job of our YOLOv3 implementation was done several days ago. It would be nice if you can manage your time to review the code. Also, I don't have many vid cards, so I haven't tested it with other backbones like ResNet yet. (But it should work in my design.)

Btw, I think if mm-detection is aimed at both academy and production, then it should contain both light-weight and heavy-weight backbones / models. Would you like me to add YOLOv3-tiny, after YOLOv3 is merged?