Mipnerf export error

[hanjoonwon]

When i try to export poisson from MipNeRF i got an error Could not find rgb in the model outputs. Please set --rgb_output_name to one of: dict_keys(['rgb_coarse', 'rgb_fine', 'accumulation_coarse', 'accumulation_fine', 'depth_coarse', 'depth_fine'])

[MatthewDZane]

I am encountering the same error when exporting the point cloud. I also ran into a similar (I assume related) error when trying to view the model in the viewer File "/home/miniconda3/envs/nerfstudio/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/mnt/c/Users/source/repos/NeRFs/nerfstudio/nerfstudio/viewer/render_state_machine.py", line 218, in run outputs = self._render_img(action.camera_state) File "/mnt/c/Users/source/repos//NeRFs/nerfstudio/nerfstudio/viewer/render_state_machine.py", line 189, in _render_img pts = camera_ray_bundle.directions * outputs["depth"] KeyError: 'depth'

[DrawingProcess]

I have a same issue, when I ns-viewer mipnerf model config.yml

$ ns-viewer --load-config outputs/headset_alphabg/mipnerf/2024-04-06_064554/config.yml
...
Traceback (most recent call last):
  File "/home/csj000714/miniconda3/envs/nerfstudio/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/home/csj000714/repos/nerfstudio/nerfstudio/viewer/render_state_machine.py", line 222, in run
    outputs = self._render_img(action.camera_state)
  File "/home/csj000714/repos/nerfstudio/nerfstudio/viewer/render_state_machine.py", line 193, in _render_img
    pts = camera_ray_bundle.directions * outputs["depth"]
KeyError: 'depth'

[zatevakhin]

The same issue is observed with bionerf. It happens when connecting to a viewer.

ns-viewer --load-config outputs/images/bionerf/2024-05-19_021853/config.yml                                                                                                                    
[11:23:57] Auto image downscale factor of 4                                                 nerfstudio_dataparser.py:484                                                                                                               
Setting up training dataset...                                                                                                                                                                                                         
Caching all 2212 images.                                                                                                                                                                                                               
Warning: If you run out of memory, try reducing the number of images to sample from.                                                                                                                                                   
Setting up evaluation dataset...                                                                                                                                                                                                       
Caching all 245 images.                                                                                                                                                                                                                
Loading latest checkpoint from load_dir                                                                                                                                                                                                
✅ Done loading checkpoint from outputs/images/bionerf/2024-05-19_021853/nerfstudio_models/step-000025999.ckpt                                                                                                                         
╭─────────────── viser ───────────────╮                                                                                                                                                                                                
│             ╷                       │                                                                                                                                                                                                
│   HTTP      │ http://0.0.0.0:7007   │                                                                                                                                                                                                
│   Websocket │ ws://0.0.0.0:7007     │                                                                                                                                                                                                
│             ╵                       │                                                                                                                                                                                                
╰─────────────────────────────────────╯                                                                                                                                                                                                
[11:24:06] disabled local writer                                                                           writer.py:185                                                                                                               
(viser) Connection opened (0, 1 total), 2655 persistent messages                                                                                                                                                                       
Exception in thread Thread-6:                                                                                                                                                                                                          
Traceback (most recent call last):                                                                                                                                                                                                     
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner                                                                                                                                                              
    self.run()                                                                                                                                                                                                                         
  File "/home/user/nerfstudio/nerfstudio/viewer/render_state_machine.py", line 224, in run                                                                                                                                             
    outputs = self._render_img(action.camera_state)                                                                                                                                                                                    
  File "/home/user/nerfstudio/nerfstudio/viewer/render_state_machine.py", line 195, in _render_img                                                                                                                                     
    pts = camera_ray_bundle.directions * outputs["depth"]                                                                                                                                                                              
KeyError: 'depth'

[DrawingProcess]

Once you enter a value of one of the outputs dict (mipnerf dict_keys is ['rgb_coarse', 'rgb_fine', 'accumulation_coarse', 'accumulation_fine', 'depth_coarse', 'depth_fine']), you can see the view.

pts = camera_ray_bundle.directions * outputs["depth_coarse"]

[DrawingProcess]

@jb-ye If you add the following to all models to have outputs["depth"], I think this problem will be eliminated.

Even if it's done this way outputs["depth"] = outputs["depth_coarse"]