Could results be different between this repo's MDv5 and the older MD repo MDv5?

[VLucet]

Search before asking

• [x] I have searched the Pytorch-Wildlife issues and found no similar bug report.

Question

Would it be possible for detection results of MDv5 from this repo to be different from the results produced by https://github.com/agentmorris/MegaDetector ?

Additional

Because providing a minimally repeatable example would be quite difficult (as one needs to do the setup required by both repositories), I thought I would ask whether it is possible to get two different results from the two implementation of the predictions from MD in the two repos. I have this repo's version set up on a VM and it seems to give me different results from the other repo's results.

[danielaruizl1]

Hi @VLucet The models for MDv5 in both repositories are the same and therefore the results should be comparable. However, we are currently validating all our results because other users have also reported inconsistencies for the results we have reported for MDv5. We will be updating this information soon, so stay tuned for our announcements!

Additionally, we would appreciate it if you could provide more details on the differences you have found between the results from both repositories. Thanks for helping us improve!

[VLucet]

Thanks for answering @danielaruizl1. I seem to be getting different scores for the bounding boxes. I seem to get the same detections, but with different scores. I will attempt to small make a docker image with both environments and a simple script that demonstrates this. In the meantime, here is an example of the differences:

This image, generated from the code in https://github.com/agentmorris/MegaDetector, gives me a score of 0.4

The same image in your gradio demo, gets a score of 0.3.

[danielaruizl1]

Hi @VLucet thanks for the example. Could you share the input image with us so we can test it on our end?