detect objects with affordance descriptions

[caoyunkang]

Hi! Thanks for your admiring work! I would like to know if there are any methods to defect objects with affordance descriptions. For example, we want to detect 'the cat on the table'. Thus, we want to detect the cat rather than the table. However, current Grouded DINO would extract all nouns in the prompts first and seemingly will extract 'cat' and 'table', respectively.

[SlongLiu]

thanks for your question. groundingdino can finish this as it is a grounding model.

The model will output multiple objects. Each object has a text similarity over text tokens. You can use a text mask to filter out the boxes with highest similarities with token 'cat'.

You may need to add extra codes to finish this for now.

Thanks for your questiones. We will make the solution more user-friendly soon.

[caoyunkang]

I understand. Thanks for your reply!

[YanhaoZhang]

@SlongLiu Hi, thanks for this awesome work. I find the segmentation results of multiple object is not as accurate than the single object segmentation. May I know for any suggestions for this? Thanks.