Visible Mask - Data generation

[utsavrai]

Often it is required to obtain only the visible portion of the mask for the object and thus removing the mask of distractors. How to achieve this?

[TontonTremblay]

I am not sure I am following, could share an example?

[utsavrai]

Sure, for example in the below case I want ground truth mask of "white watering can" behind the yellow duck, then the output mask should be only the visible portion of the "white watering can". Similarly when segmentation mask is generated then it would be great to have an option which only outputs the visible portion of the object and excluding all the distractor mask.

[TontonTremblay]

https://github.com/owl-project/NVISII/blob/master/examples/09.meta_data_exporting.py#L125-L139 check this out, this is what you want to save, if you look at the code, when we compute the visibility, we move objects and render them around to get percentage.

[utsavrai]

Hi @TontonTremblay, if possible could you please provide a simple example to achieve this. Thanks

[TontonTremblay]

Did you look at the code? I pointed at directly what you need.

[utsavrai]

Hi, I did look at the code

# the entities are stored with an id, 
# nvisii.entity.get_id(), this is used to 
# do the segmentation. 
# ids = nvisii.texture.get_ids_names()
# index = ids.indexof('soup')
# nvisii.texture.get('soup').get_id()
nvisii.render_data_to_file(
    width=opt.width, 
    height=opt.height, 
    start_frame=0,
    frame_count=1,
    bounce=int(0),
    options="entity_id",
    file_path = f"{opt.outf}/entity_id.exr"
)

what I don't understand is how do I manipulate the visibility of the segmentation mask. In addition to this if I want to exclude a certain entity from the segmentation mask how do I achieve it? The render_data_to_file with enitity_id as an option renders the segmentation mask of all the entities. Is there a way to exclude a certain entity and also control the visibility? Thanks

[TontonTremblay]

https://github.com/TontonTremblay/nvisii_mvs/blob/main/utils.py#L2050

The idea is, you get the segmentation mask, which has entity ids in there for your objects. You remove anything else that is not your object id. Then you have your mask. np.where should work just fine for that. Here is an example that I used to produce percentage visibility of an object where I render the scene with only the object, then I render the object with the other objects and compare the pixels.