VGT evaluation result not matching for DoclayNet.

[soeb-hussain]

trafficstars

Hi,

Thank you for creating and sharing the Vision Grid Transformer repository.

I am currently trying to evaluate the model on the DocLayNet test dataset in order to replicate the published results (mAP of 83.7). I am using the weights available here: doclaynet_VGT_model.pth.

I executed the evaluation using the following command:

bash Copy code python path/to/train_VGT.py --config-file VGT/object_detection/Configs/cascade/doclaynet_VGT_cascade_PTM.yaml --eval-only --num-gpus 1 MODEL.WEIGHTS VGT/downloads/weights/doclaynet_VGT_model.pth OUTPUT_DIR VGT/AdvancedLiterateMachinery/DocumentUnderstanding/VGT/downloads However, the results I obtained differ from the published ones. Please see the attached matrix for reference:

Could you please advise if I might be overlooking something in the evaluation process?