Result mismatch on OV-COCO

[Lukas-Ma1]

Hi! Appreciated for your outstanding work! I try to reproduce your experiment. I download your model weights COCO_OVD_RKD_PIS_WeightTransfer and evaluate it. But I got 31.61 AP, which is far away from the result in paper. Here is my file structure and output log. I don't understand why. My python is 3.8.19, and here is my version of python package: Package Version Editable project location

---

absl-py 2.1.0 antlr4-python3-runtime 4.9.3 black 24.4.2 bleach 6.1.0 cachetools 5.3.3 certifi 2024.2.2 charset-normalizer 3.3.2 click 8.1.7 clip 1.0 cloudpickle 3.0.0 contourpy 1.1.1 cycler 0.12.1 Cython 3.0.10 detectron2 0.6 /data/psz/object-centric-ovd/detectron2 fonttools 4.51.0 ftfy 6.2.0 fvcore 0.1.5.post20221221 google-auth 2.29.0 google-auth-oauthlib 1.0.0 grpcio 1.63.0 hydra-core 1.3.2 idna 3.7 importlib_metadata 7.1.0 importlib_resources 6.4.0 iopath 0.1.9 joblib 1.4.2 kaggle 1.6.12 kiwisolver 1.4.5 lvis 0.5.3 Markdown 3.6 MarkupSafe 2.1.5 matplotlib 3.7.5 mypy-extensions 1.0.0 numpy 1.24.4 oauthlib 3.2.2 omegaconf 2.3.0 opencv-python 4.9.0.80 packaging 24.0 pathspec 0.12.1 Pillow 9.4.0 pip 24.0 platformdirs 4.2.2 portalocker 2.8.2 protobuf 5.26.1 pyasn1 0.6.0 pyasn1_modules 0.4.0 pycocotools 2.0.7 pyparsing 3.1.2 python-dateutil 2.9.0.post0 python-slugify 8.0.4 PyYAML 6.0.1 regex 2024.5.15 requests 2.31.0 requests-oauthlib 2.0.0 rsa 4.9 scikit-learn 1.3.2 scipy 1.10.1 setuptools 59.5.0 six 1.16.0 tabulate 0.9.0 tensorboard 2.14.0 tensorboard-data-server 0.7.2 termcolor 2.4.0 text-unidecode 1.3 threadpoolctl 3.5.0 tomli 2.0.1 torch 1.10.0 torchaudio 0.10.0 torchvision 0.11.0 tqdm 4.66.4 typing_extensions 4.11.0 urllib3 2.2.1 wcwidth 0.2.13 webencodings 0.5.1 Werkzeug 3.0.3 wheel 0.43.0 yacs 0.1.8 zipp 3.18.1

Moreover, I use the file from your provided link, such as MAVL_proposals, annotations in coco... By the way, can you provide the output log when you training?