facebook360_dep
facebook360_dep copied to clipboard
Feature: Semantic Segmentation in Masking
Foreground masks act as a rough proxy to objects in the scene. Significant work on semantic segmentation (re: https://github.com/facebookresearch/Detectron) has been done, although they have not been thoroughly integrated with our pipeline to improve results.
The current behavior and new feature
Render (Foreground masks): A weighted sum of a traditional foreground mask and semantic segmentation can be used to obtain a more accurate set of masks for the scene. This can even be extended to have a multi-planar image if the semantic segmentation is accurate enough.
Expected use cases
Significant research effort.