AutoRF (unofficial)

This is unofficial implementation of "AutoRF: Learning 3D Object Radiance Fields from Single View Observations", which performs implicit neural reconstruction, manipulation and scene composition for 3D object. In this repo, we use KITTI dataset.

Quick Start

Download KITTI data and here we only use image data

└── DATA_DIR
       ├── training   <-- training data
       |   ├── image_2
       |   ├── label_2
       |   ├── calib

Run the preprocess scripts, which produce instance mask using pretrained PointRend model.

python scripts/preproc.py

After this, you will have a certain directory which contains the image, mask and 3D anotation of each instance.

└── DATA_DIR
       ├── training
       |   ├── nerf
           |   ├── 0000008_01_patch.png
           |   ├── 0000008_01_mask.png
           |   ├── 0000008_01_label.png

Run the following sciprts to train a nerf model

python src/train.py

After training for serveral iterations (enough is ok), you can find the checkpoint file in the ``output'' folder, and then you can perform scene rendering by running

python src/train.py --demo

Notice

You can adjust the manipulaion function (in kitti.py) by your self, here I only provide the camera pushing/pulling and instance rotation.