How do you pre-render the Replica Dataset and how to render own Dataset

[Yiiii19]

Hi, thanks for your code and efforts! I have two questions which about pre-render the Dataset.

1. From the NeRF Dataset we can take photos and get poses through the library directly. How do you pre-render the Replica Dataset which you provided us?
2. How can we pre-render our own dataset for using this semantic-nerf? What should I prepare my own Dataset Many thanks for your reply!

[Harry-Zhi]

Hi,

Sorry for the late reply.

1. As mentioned in the paper, I used Habita-Sim to render images, semantics using my pre-generated random poses. If you are interested in generating your own data in Replica, habitat-sim is a recommended tool to use.
2. Like NeRF, you need to have posed RGB images with some amount of semantic labels, which can either be dense labels from existing dataset or CNN predictions. For ScanNet experiments, I directly used the raw scans from the dataset with provided poses and labels, for Replica I used Habitat-sim to render random sequences. For other datasets if you have proper posed images and labels of an indoor room-level scene, I would expect Semantic-NeRF to work in a similar manner. Hope this helps!

[thangnn123456]

Hi @Harry-Zhi ,

I got a problem when trying to run on Scannet.

The problem occurred from line 55. I checked the Scannet, and can not find "renders", "pose", "color", "depth" folders.

Could you help? Or could you upload the pre-rendered of Scannet?

[HongweiXie]

Hi @Harry-Zhi ,

I got a problem when trying to run on Scannet.

The problem occurred from line 55. I checked the Scannet, and can not find "renders", "pose", "color", "depth" folders.

Could you help? Or could you upload the pre-rendered of Scannet?

I had the same problem. Have you figured out？

[Harry-Zhi]

Hi all.

Actually I put all the parsed scannet images per scenes into a unified folder called "renders", where the subfolders "pose", "color", "depth" contains the corresponding data.

I have added the modified scannet parser and please feel free to change it for your own purpose: https://github.com/Harry-Zhi/semantic_nerf/blob/main/SSR/datasets/scannet/scannet_reader.py

The scritps are taken from ScanNet python reader https://github.com/ScanNet/ScanNet/tree/master/SensReader/python and python2.7 is used for parsing data.

@thangnguyennngoc @HongweiXie

[derrick-xwp]

Dear authors,

Can you share the code that you used to sample from replica?

THanks

[Harry-Zhi]

Dear authors,

Can you share the code that you used to sample from replica?

THanks

Hi, I am sorry that the rendering codes are currently not released yet. However, as mentioned above, we used randomly generated camera poses and feed them into Habitat-Sim agent to render corresponding images and labels.

Following links are some good starting points which you could build you own rendering pipeline: https://aihabitat.org/docs/habitat-lab/habitat-sim-demo.html https://aihabitat.org/docs/habitat-sim/image-extractor.html

Hope they are useful and please let me know if you have any issues implementing your own version.

[HLinChen]

Dear authors, Can you share the code that you used to sample from replica? THanks

Hi, I am sorry that the rendering codes are currently not released yet. However, as mentioned above, we used randomly generated camera poses and feed them into Habitat-Sim agent to render corresponding images and labels.

Following links are some good starting points which you could build you own rendering pipeline: https://aihabitat.org/docs/habitat-lab/habitat-sim-demo.html https://aihabitat.org/docs/habitat-sim/image-extractor.html

Hope they are useful and please let me know if you have any issues implementing your own version.

Hi,

Thank you for the wonderful work. The images rendered seem to be in a smooth trajectory instead of produced from random camera poses. May I ask what is the algorithm used to generate the trajectory?

Thanks!