Question: how to get the whole 2d bbox of an asset

[calingc]

Hi guys, I hope you're doing well.

I have the following question. Can unity perception output the 2d bbox of an object, without taking into account how much of it is visible? I noticed that the in-built 2d bbox labeler outputs the 2d bbox only around the visible part of an object of interest. However I'm interested in the whole 2d bbox as if the object was not occluded by anything.

Reason is that we're interested in the visibility percentage of an object: mask area / total 2d bbox area. We need this to filter out annotations in which the object is not visible enough: similar to how human annotators will not annotate everything in a scene.

I guess my questions are:

• Can the perception package output the total 2d bbox?
• Even better is there a way to extract directly the visibility percentage of an object of interest?

Many thanks!

[esgnn]

Unfortunately, this is not available right now https://github.com/Unity-Technologies/com.unity.perception/issues/401

[calingc]

Got it, thanks for the quick reply! Is there any plan on releasing the feature and if so, could you provide a timeline for that?

Alternatively, do you have any suggestions on how to implement a custom labeler that does that? in order to extract the full 2d bbox I guess I just need to iterate over all its vertices retain the min max coords and transform their position to camera space. However, do I have access to that type of info inside a labeler?

[esgnn]

I am sorry if I misled you, I am not an employee. Not sure if you have that info inside a labeler. If I am not mistaken a new version is coming, maybe there's a feature to have that info.

[RuiyuZ]

Thanks @esgnn for the reply! Hi @calingc, right now it's not really possible to do it that way. I think you can attempt to approximate it with either the 3D bounding box or maybe the keypoint labeler, like what was discussed in #401

[esgnn]

@RuiyuZ, no problem at all. Will it be supported in upcoming versions? Thank you.

[RuiyuZ]

Hi @esgnn, It is not included in this upcoming version (which will be released in a few days), but I will mark this as a feature request and report it to the team, thanks!

[JonathanHUnity]

Thanks for reaching out about this feature - I'll call it 'occlusion-independent bounding boxes'. Please drop an email to @shounakmitra at [email protected] to chat more about this. He is gathering feature requests and can share more about our roadmap.

[StevenBorkman]

Hey all! The Perception team just released 🎉 Perception 1.0 🎉, a major update to the toolset! With this release we have introduced a new occlusion labeler which will provide the occluded amount (as a percentage) to all labeled objects.

There are a bunch of cool features to check out such as ray tracing, depth, a new output format SOLO, normals, and more! Here is the full list of changes: Perception 1.0 Changelog