How to render instance id segmentation masks

[DianCh]

❓ Questions on how to use PyTorch3D

Hi! Thank you for maintaining this wonderful repo! Let's say I have meshes of objects, with their instance ids (or semantic category ids), how can I render the 2d masks of them, with the pixels storing the ids?

Thank you in advance! I'd really appreciate some suggestions.

[bottler]

You can do this with normal rendering where the channel is the id rather than the color. This works and has been used, although it can use a lot of memory. For the lights, you'll need to use AmbientLights with an explicit ambient_color.

[zhifanzhu]

I have just implemented this method. My idea is to keep a record of which face corresponds to which mesh, and replace fragments.pix_to_face with faceid_to_meshid[pix_to_face]. Here's my implementation for anyone interested:

https://github.com/zhifanzhu/libzhifan/blob/master/libzhifan/geometry/instance_id_rendering.py https://github.com/zhifanzhu/libzhifan/blob/master/tests/geometry/projection_test.py#L201-L241

simple example output