About Train Detr by custom dataset and custom backbone

[wangcn99]

I am aware that you are trying to replace the backbone network of the DETR. Now,I also do the same work,but i havent get good result. Do you have any good suggestions? I think I didnt use good learning rate or other hyper paramters I use CBAM replace resnet thanks :)

[lessw2020]

Hi @wangcn99 - thanks for the update re: CBAM. I suspect there is more linkage between the DETR head and the backbone than initially thought. Thus, swapping in new backbones tends to be much more disruptive to DETR. I tried mobilenet and ResNeST, and neither did very well, so currently I still use the default Resnet-50.

One thing you could try is using AdaBelief optimizer instead of AdamW. I'm using it with GAN's right now and it's performing nicely.
That should give you better results than training with Adam(W).

You can use their hyperparameters - adabelief is here: https://github.com/juntang-zhuang/Adabelief-Optimizer

[nikky4D]

@lessw2020 Have you tried any new experiments with switching out detr's backbone? Did you get it to work or it is still problematic with other backbones?

Also, instead of changing the backbone, do you have any experience with changing the weights for the resnet50 which i suspect should be useful when dealing with non-imagenet domains.