Model finds the objects that are not there in reality

[Kamipedia]

Dear authors, Thank you so much for your amazing work. In the process of using your model I realized that in case I write a prompt with the object that is not present in the image (i.e I tried finding dinosaur in the picture with pizzas, it found the the dinosaur that is not present in the picture). It seems like the model was not trained on negative examples. Can you, please, resolve this issue as it will significantly increase the usability of your model.

Thanks in advance!

[CoderZhangYx]

You are right. There were no negative examples in training datasets. We are looking for datasets containing negative examples. Currently we only find G-RefCOCO containing a few negative examples but its other labels are poor, leading to performance decrease. We will try to annotate some data through foundation models. Thank you for your nice advice!

[Riku-tohoku0224]

I am having trouble in the same place. Has there been any progress since then? Thank you for your excellent work.

[CoderZhangYx]

We've employed G-RefCOCO to joint train our model but find that such cases still exist. Honestly it is a shared problem of existing RES models because of limited negative examples in availabel datasets. Currently we are focusing on extending model's ability in more segmentation areas and would deal with the recall problem in the long future.