CrossStagePartialNetworks icon indicating copy to clipboard operation
CrossStagePartialNetworks copied to clipboard

Did you compare AP and FPS (rather than BFLOPS) of models ?

Open AlexeyAB opened this issue 4 years ago • 39 comments

@WongKinYiu Hi,

Did you compare AP (MS COCO) and 1080ti fps (rather than BFLOPS) of models ?

  • CSPResNeXt50-PANet-SPP-GIoU https://github.com/WongKinYiu/CrossStagePartialNetworks#gpu-real-time-models

  • CSPDarkNet-53-SPP-GIoU-Yolov3 that is based on CSPDarkNet-53 https://github.com/WongKinYiu/CrossStagePartialNetworks#big-models

  • default Yolov3-SPP model

With the same: network resolution, mini_batch=batch/subdivisions, data augmentation, the same dataset - to compare apples with apples.

AlexeyAB avatar Nov 28 '19 12:11 AlexeyAB

I found: https://arxiv.org/abs/1911.11929v1

But did you try backbones CSPDarkNet-53-SPP and CSPDarkNet-53-Elastic for Yolov3-spp / PAN?

image

AlexeyAB avatar Nov 28 '19 12:11 AlexeyAB

@AlexeyAB Hello,

Yes, we do comparison of AP, fps, BFLOPs, #parameters. image

For big models, due to I have only few GPUs. Training one model need about one and half month. So currently we only have results of CSPResNeXt50-PANet-SPP and CSPResNeXt50-PANet-SPP-GIoU. If you would like to test the speed of other models, I could upload the cfg files.

WongKinYiu avatar Nov 28 '19 12:11 WongKinYiu

This result is for CSPResNeXt50-PANet-SPP, not CSPResNeXt50-PANet-SPP-GIoU. We focus on AP50, so in our paper, we do not use GIoU. image

WongKinYiu avatar Nov 28 '19 12:11 WongKinYiu

@AlexeyAB Hello,

I upload the cfg files to in progress. Please fell free to try them.

WongKinYiu avatar Nov 28 '19 12:11 WongKinYiu

@WongKinYiu Thanks!

Will you release cfg/weights files for these models?

image

AlexeyAB avatar Nov 28 '19 12:11 AlexeyAB

@AlexeyAB Hmm...

Due to the non-disclosure agreement, I can not release the backbones of lightweight models currently. If you are interested in the EFM(SAM) or PRN, I can share you the head cfg of these models. Well...next month I will try to discuss with the company whether I can release some models or not.

WongKinYiu avatar Nov 28 '19 12:11 WongKinYiu

Hi @WongKinYiu

Now it ’s "Next Month".

LOL

Waiting for you share.

Code-Fight avatar Dec 09 '19 01:12 Code-Fight

@Code-Fight Tomorrow i ll discuss with them.

WongKinYiu avatar Dec 09 '19 02:12 WongKinYiu

@WongKinYiu Great Looking forword your good news.

Code-Fight avatar Dec 09 '19 02:12 Code-Fight

falid :sob: :sob: :sob:

WongKinYiu avatar Dec 10 '19 07:12 WongKinYiu

oh....too sad, but thanks for your best

Code-Fight avatar Dec 10 '19 07:12 Code-Fight

@WongKinYiu Hi

I would like to test csresnet50-elastic-panet-spp.cfg, csresnext50-yolo-spp.cfg, csresnext50-panet-spp.cfg, csresnext50-panet-spp-giou.cfg on gtx1080ti

What should i change in config files for classes = 2 and width=640 height=384

Is it enough to:

  1. change line classes=80 to 2 in each of 3 [yolo]-layers
  2. change [filters=255] to filters=(2+ 5)x3 in the 3 [convolutional] before each [yolo] layer as in @AlexeyAB documentation?

Is it possible to use darknet53.conv.74 as initial weight? Or train from scratch?

Thank you.

dreambit avatar Dec 10 '19 11:12 dreambit

@dreambit hello,

plz use the pretrained weights from https://github.com/WongKinYiu/CrossStagePartialNetworks#big-models

and follow the https://github.com/AlexeyAB/darknet#how-to-train-tiny-yolo-to-detect-your-custom-objects to get initial weights like csresnext50.conv.80 and so on.

WongKinYiu avatar Dec 10 '19 12:12 WongKinYiu

@WongKinYiu Thanks, what about changes in config files with different classes and network size? are there any changes that have to be made except those mentioned in @AlexeyAB docs?

dreambit avatar Dec 10 '19 13:12 dreambit

@dreambit

change line classes=80 to 2 in each of 3 [yolo]-layers change [filters=255] to filters=(2+ 5)x3 in the 3 [convolutional] before each [yolo] layer as in @AlexeyAB documentation?

it is enough.

WongKinYiu avatar Dec 10 '19 13:12 WongKinYiu

@WongKinYiu Hello

I have almost completed training csresnext50-panet-spp-giou.cfg. Could u provide (if you have one) elastic version of csresnext50-panet-spp[-giou], so i can test it.

Thank you.

dreambit avatar Dec 17 '19 10:12 dreambit

@dreambit

https://github.com/WongKinYiu/CrossStagePartialNetworks/blob/master/in%20progress/csresnext50-elastic-panet-spp.cfg here u r.

WongKinYiu avatar Dec 17 '19 11:12 WongKinYiu

@WongKinYiu, appreciate it 👍

dreambit avatar Dec 17 '19 11:12 dreambit

@WongKinYiu I've changed network size to 640x385, and classes = 2 filters = 21(in each conv. layer before yolo), random = 0

Bug i have 0 mAP.

What is wrong?

image

Thanks.

dreambit avatar Dec 17 '19 16:12 dreambit

@dreambit

Use pre-trained weights.

Set 608x384

https://github.com/AlexeyAB/darknet#how-to-improve-object-detection

increase network resolution in your .cfg-file (height=608, width=608 or any value multiple of 32) - it will increase precision

AlexeyAB avatar Dec 17 '19 20:12 AlexeyAB

@AlexeyAB

It was a typo, size is 640x384

@WongKinYiu @AlexeyAB I used wrong initial weights, now it is ok.

dreambit avatar Dec 18 '19 07:12 dreambit

@dreambit great!

WongKinYiu avatar Dec 18 '19 07:12 WongKinYiu

I have changed network size to 608x608, classes = 40, and filters = 135 (in each conv. layer before yolo), random = 1. I have used csresnext50-panet-spp.cfg and csresnext50c.conv.80 to train my own data set. However, current avg loss = -nan and 0 mAP. What is the problem?

chart

cmtsai avatar Dec 19 '19 11:12 cmtsai

@cmtsai Hello,

csresnext50-panet-spp.cfg should use csresnext50.conv.80 as pre-trained model, not csresnext50c.conv.80.

WongKinYiu avatar Dec 19 '19 12:12 WongKinYiu

@WongKinYiu Hello, 3Q very much for your response! How to obtain csresnext50.conv.80?

cmtsai avatar Dec 20 '19 01:12 cmtsai

@WongKinYiu Hi, I got it! 3Q very much!

cmtsai avatar Dec 20 '19 01:12 cmtsai

@WongKinYiu Hi,

Did you try to use Maxout for CSP?

I.e. after route use maxpool_depth:

[route]
layers = 36, 21

[maxpool]
maxpool_depth=1
out_channels=256
stride=1
size=1

image

AlexeyAB avatar Dec 25 '19 01:12 AlexeyAB

@AlexeyAB

Hello, I used maxout in small model. and I applied maxpool_depth before route.

WongKinYiu avatar Dec 25 '19 01:12 WongKinYiu

@WongKinYiu Hi,

Why did you add CenterNet resnet-102 instead of CenterNet dla-34 to your comparison table?

AlexeyAB avatar Dec 30 '19 13:12 AlexeyAB

@AlexeyAB

Hello, we test CenterNet dla-34 on 1080ti, but it can not reach > 30 fps. In my test, CenterNet resnet-102 run faster than CenterNet dla-34 almost twice times. So I only compare with CenterNet resnet-102.

There are also many issues of inference speed of dla on CenterNet's github. I have not found solution to solve the inference time problem. If you face the problem and solve it, please tell me how to deal with it.

WongKinYiu avatar Dec 30 '19 13:12 WongKinYiu