Change of backbone

[pinchazer]

Hello @herobd Thanks for your awesome paper, it was something new for this kind of problem! However I'm curious about backbone that you used in this work. YOLOv2 seems outdated nowadays so what is your thoughts about adopting YOLOv5 architecture for this task? Is it good idea or not? I have a desire to reproduce your idea but with new backbone and wanted to hear what do you think about it?

Thanks in advance !

[herobd]

You should be able to replace the backbone without any issues as the output needed is just the class and bounding boxes (you could even replace it even with a RCNN type detector). I do take the features from two places in the backbone for the graph network, but that's pretty easy to tweak for a new backbone.

[Ryadm]

Hello @herobd Thank you very much for your interesting paper ! I am also interested in changing the backbone and I would like to replace it by a Faster RCNN detector. I already trained it on FUNSD for the questions/answers/headers/other detection and classification. However, I have also to replace the features ("saved_features" in your implementation) by new features, and I wanted to know if you have some ideas about how I can replace these features by new ones from my FRCNN detector. (I am using a Resnet50-FPN as backbone for the FasterRCNN)

Thank you very much !

[herobd]

It should be pretty easy. For the detector, just register a forward hook on the layer you want that saves the features in detector.saved_features. This happens in setForGraphPairing() in yolo_box_detector.py (yours can be much simpler, this is complicated because I tested pulling various different layers).

You then just need to be sure the scale is set correctly for the roi_align. This is read from detector.save_scale which I also set in setForGraphPairing(), but you could hard code. I'd recommend starting off saving just one layer for features, as more doesn't improve things significantly.

[Ryadm]

It should be pretty easy. For the detector, just register a forward hook on the layer you want that saves the features in detector.saved_features. This happens in setForGraphPairing() in yolo_box_detector.py (yours can be much simpler, this is complicated because I tested pulling various different layers).

You then just need to be sure the scale is set correctly for the roi_align. This is read from detector.save_scale which I also set in setForGraphPairing(), but you could hard code. I'd recommend starting off saving just one layer for features, as more doesn't improve things significantly.

Thank you so much !

[harshsummit]

hey @Ryadm can you push your code to a branch or create a repo with new code?