nerfstudio
nerfstudio copied to clipboard
Adding Terminal Window to Jupyter Cells so all Colab users (free and pro) can train with Nerfstudio
Background and Issues:
- All colab users (free and pro) were not be able to run
ns-train
in a jupyter cell, and this has been made apparent in several issues below. (Pro users can get around this, but it's still problematic and explained below) - https://github.com/nerfstudio-project/nerfstudio/issues/2283
- https://github.com/nerfstudio-project/nerfstudio/issues/1836
- https://github.com/nerfstudio-project/nerfstudio/issues/1832
- https://github.com/nerfstudio-project/nerfstudio/issues/1113
- https://github.com/nerfstudio-project/nerfstudio/issues/2166
The culprit of this bug is that as of Feb 2024, Google Colab runs python 3.10, which has an issue with nerfstudio's rich console outputs. In particular, a python module called shutil
will set the terminal size to 0 when dealing with rich console outputs. As a result, nothing gets printed, and Colab will shutdown the cell and terminate early before training can begin.
This is mentioned in other python modules which use the rich console output, and a complete breakdown on the issue is detailed here.
Overview of Changes
- By using the
colab-xterm
module, we are able to create a terminal window in a jupyter cell output. - After installing it, we clear the cell output to make it very user-friendly to follow the instructions.
- The command is printed and instructions are given to help the user run
ns-train
and they will be able to generate the viser/viewer link to see the training progress - Removed the "Connecting to Viewer Cell" as users can simply use
ns-train
's sharable URL link - Final change was to add the torchaudio dependency which was mentioned here so that users can train.
Impact and Other Notes
-
While Pro Colab users can get around this by running
ns-train
in their terminal window, Colab only allows for 1 terminal window. Therefore, to run the render command, pro users need to Ctrl+C the training process hosting the viser/viewer link, which can be quite irritating - especially if they want to make more than just 1 render path. -
Therefore, even once Colab upgrades to python 3.11 in April 2024, there is still a need to have at least 2 terminal windows, which is possible with this change.
-
To run
ns-train
, users (free and pro) simply need to follow the instructions to copy and paste the command into the terminal window. This is pretty clear to follow, and looks like the following: -
Below is what the cell terminal window looks like after copy+pasting the command on a free colab account:
Just found this as a fix to my own issue with getting the colab notebook to work. This does look to be working (I'm currently at 5% trained on one of the demo scenes), but I did notice you had a little error in the imports. There is now an extra " on the end of an import line. Just wanted to point this out and the non-blocking error it causes before you try to merge this
@jfenton888 thank you for the catch! I've updated the PR with this change
Thank you!!! Going to test this soon and see if anything is needed, but it looks like everything is good!