NameError: name 'tcnn' is not defined when trying to train nerfacto

[dokluch]

Describe the bug I get a NameError: name 'tcnn' is not defined error right after starting to train nerfacto model Vanilla model is trained without issues though

[visualizer.py:96] Sending ping to the viewer Bridge Server...
[visualizer.py:98] Successfully connected.
[WARNING] Not running eval iterations since only viewer is enabled. Use `--vis wandb` or `--vis tensorboard` to run
with eval instead.
disabled tensorboard/wandb event writers
Auto image downscale factor of 2
[nerfstudio_dataparser.py:91] Skipping 0 files in dataset split train.
Auto image downscale factor of 2
[nerfstudio_dataparser.py:91] Skipping 0 files in dataset split val.
Loading data batch ---------------------------------------- 100% 0:00:00
Loading data batch ---------------------------------------- 100% 0:00:00
[profiler.py:85] Printing profiling stats, from longest to shortest duration in seconds
Traceback (most recent call last):
  File "C:\Users\user\miniconda3\envs\nerfstudio\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\user\miniconda3\envs\nerfstudio\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\user\miniconda3\envs\nerfstudio\Scripts\ns-train.exe\__main__.py", line 7, in <module>
  File "C:\Code\nerfstudio\scripts\train.py", line 216, in entrypoint
    main(tyro.cli(AnnotatedBaseConfigUnion))
  File "C:\Code\nerfstudio\scripts\train.py", line 202, in main
    launch(
  File "C:\Code\nerfstudio\scripts\train.py", line 145, in launch
    main_func(local_rank=0, world_size=1, config=config)
  File "C:\Code\nerfstudio\nerfstudio\engine\trainer.py", line 62, in train_loop
    trainer.setup()
  File "C:\Code\nerfstudio\nerfstudio\engine\trainer.py", line 129, in setup
    self.pipeline: VanillaPipeline = self.config.pipeline.setup(
  File "C:\Code\nerfstudio\nerfstudio\configs\base_config.py", line 62, in setup
    return self._target(self, **kwargs)
  File "C:\Code\nerfstudio\nerfstudio\pipelines\base_pipeline.py", line 220, in __init__
    self._model = config.model.setup(
  File "C:\Code\nerfstudio\nerfstudio\configs\base_config.py", line 62, in setup
    return self._target(self, **kwargs)
  File "C:\Code\nerfstudio\nerfstudio\models\base_model.py", line 80, in __init__
    self.populate_modules()  # populate the modules
  File "C:\Code\nerfstudio\nerfstudio\models\nerfacto.py", line 104, in populate_modules
    self.field = TCNNNerfactoField(
  File "C:\Code\nerfstudio\nerfstudio\fields\nerfacto_field.py", line 123, in __init__
    self.direction_encoding = tcnn.Encoding(
NameError: name 'tcnn' is not defined

Windows 10 Nvidia A6000

[akristoffersen]

Hi, this generally means that tinycuda didn't get installed correctly.

As a test, if you open up an interactive python environment, does importing tinycudann work? If not, this means that the conda environment hasn't installed tinycuda correctly.

(nerfstudio) $ python3
> import tinycudann

[dokluch]

@akristoffersen just found out that tinycudann fails to install and to compile on my machine. Will try to investigate.

[stevekm]

I know this one is closed but I am kinda wondering why tinycudann is not included with the installation setup (or at least the docs) if its required for usage?

[stevekm]

sorry looks like I was mistaken, I think the issue is actually related to this causing the CUDA install to break; https://github.com/nerfstudio-project/nerfstudio/issues/739

[THE-COB]

Hi, this generally means that tinycuda didn't get installed correctly.

As a test, if you open up an interactive python environment, does importing tinycudann work? If not, this means that the conda environment hasn't installed tinycuda correctly.

(nerfstudio) $ python3
> import tinycudann

Hi sorry I'm new here (so my issue might be dumb) and I'm working on install. I'm getting the same error as stated in the issue, but I think I was able to install tiny cuda nn using the instructions in readme. This runs perfectly fine:

(nerfstudio) PS C:\Users\rohan\Documents\fa22\cs194\nerfystuff> python
Python 3.8.15 (default, Nov 24 2022, 14:38:14) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tinycudann
>>>

Any ideas?

EDIT: The problem was fixed when I built from source! Does it mean some new version just needs to be pushed to pip or whatever?

[mgupta70]

Hi, this generally means that tinycuda didn't get installed correctly. As a test, if you open up an interactive python environment, does importing tinycudann work? If not, this means that the conda environment hasn't installed tinycuda correctly.

(nerfstudio) $ python3
> import tinycudann

Hi sorry I'm new here (so my issue might be dumb) and I'm working on install. I'm getting the same error as stated in the issue, but I think I was able to install tiny cuda nn using the instructions in readme. This runs perfectly fine:

(nerfstudio) PS C:\Users\rohan\Documents\fa22\cs194\nerfystuff> python
Python 3.8.15 (default, Nov 24 2022, 14:38:14) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tinycudann
>>>

Any ideas?

EDIT: The problem was fixed when I built from source! Does it mean some new version just needs to be pushed to pip or whatever?

I am also encountering the same error and have spent 3 days in reinstalling and breaking everything. I have CUDA 11.7 with torch 1.12.1+cu113 and torchvision 0.13.1+cu113. The command 'import tinycudann' works. I get the same error: NameError: name 'tcnn' is not defined. Can anyone please explai n the steps. What should I build from source - Tinycudann or nerfstudio?

[tancik]

It seems like it must be using the wrong environment. You could try removing the try except code here - https://github.com/nerfstudio-project/nerfstudio/blob/fad5a839aac0e51e326173d65144a726b358b514/nerfstudio/fields/nerfacto_field.py#L51-L55 To just import tinycudann as tcnn to see what error is raised. I assume some error must be getting raised since tcnn is not available later in the method.

[mgupta70]

Thanks for the prompt reply @tancik . I already tried it. It raises an error regarding DLL file _86_C Do you know about this?

[tancik]

The image you have above shows that it can be imported correctly, what is different in that case? Also can you post the full error.

[mgupta70]

I will post it tomorrow when I will have access to it.

• On a side note Also, I noticed I had Pytorch 1.13.1 with CUDA (11.7) installed already along with torchvision 0.14.1 Then, I successfully compiled tinycudann But, when I ran 'pip install nerfstudio' then it uninstalled Pytorch 1.13.1 and installed 1.12.1 which was not GPU enabled. Do you recommend using CUDA 11.6 instead of 11.7?

[orrblue]

It seems like it must be using the wrong environment. You could try removing the try except code here -

https://github.com/nerfstudio-project/nerfstudio/blob/fad5a839aac0e51e326173d65144a726b358b514/nerfstudio/fields/nerfacto_field.py#L51-L55

To just import tinycudann as tcnn to see what error is raised. I assume some error must be getting raised since tcnn is not available later in the method.

I also get this issue after following the installation instructions for Torch 2.0.1 with CUDA 11.8. My machine is running Ubuntu 20.04 with a GeForce GTX 1650 Mobile / Max-Q graphics card. To isolate the issue, in my conda environment (nerfstudio), I opened python and attempted import tinycudann. And I get ImportError: /home/hcilab/.local/lib/python3.8/site-packages/tinycudann_bindings/_75_C.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda20CUDACachingAllocator9allocatorE

[orrblue]

I will post it tomorrow when I will have access to it.

* On a side note
  Also, I noticed I had Pytorch 1.13.1 with CUDA (11.7) installed already along with torchvision 0.14.1
  Then, I successfully compiled tinycudann
  But, when I ran 'pip install nerfstudio' then it uninstalled Pytorch 1.13.1 and installed 1.12.1 which was not GPU enabled. Do you recommend using CUDA 11.6 instead of 11.7?

Following @mgupta70 's note about torch versions changing at some point during the install process, I did conda list | grep torch and saw that my torch version is 1.13.1 . Then, I updated torch with pip install torch --upgrade and it's now at version 2.0.1. Then, I re-ran ns-train nerfacto --data <path_to_data> and it now overcomes the issue with importing tinycudann. Upgrading torch seems to have fixed this particular issue!

But a new issue has emerged: File "/home/hcilab/miniconda3/envs/nerfstudio2/lib/python3.8/site-packages/torch/cuda/amp/grad_scaler.py", line 372, in step assert len(optimizer_state["found_inf_per_device"]) > 0, "No inf checks were recorded for this optimizer." AssertionError: No inf checks were recorded for this optimizer.

[jkulhanek]

Can you try to install nerfstudio from the main branch? You can do that by running pip install git+https://github.com/nerfstudio-project/nerfstudio.git

[orrblue]

That seems to work, thank you! The training ETA on data/nerfstudio/poster for me is 1 hour, 20 minutes on my laptop's GeForce GTX 1650 Mobile / Max-Q graphics card. Seems slow, but I'll look into it...

[jkulhanek]

Is it slower then the previous version?

[orrblue]

Nope, just thought it would be faster. It's my first time--sorry for any confusion. Now I'm having issues with viewing the rendered scene, so I posted a reply to another issue (765, but not linking it as it's unrelated)