SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field

Project Page | Video | Paper

SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field

[Chong Bao, Yinda Zhang^Co-Authors,Bangbang Yang^Co-Authors], Tianxing Fan, Zesong Yang, Hujun Bao, Guofeng Zhang, Zhaopeng Cui.

CVPR 2023

⚠️ Note: This is only a preview version of the code. Full code (with training scripts) will be released soon.

Installation

We have tested the code on Python 3.7.0 and PyTorch 1.10.1, while a newer version of pytorch should also work. The steps of installation are as follows:

• create virtual environment: conda create --name sine python=3.7 and activate environment
• install required python packages by bash install.sh

Data

We provide the poses of each dataset for evaluation.

Evaluation

All the pre-trained models and configs can be found here.

You can evaluate images with the pre-trained models.

python eval.py \
    --config configs/texture/vasedeck_snowy.yaml \
    --ckpt_path checkpoints/texture/vasedeck_snowy/latest.ckpt \
    --split test_train 

Citing

@inproceedings{bao2023sine,
    title={SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field},
    author={Bao, Chong and Zhang, Yinda and Yang, Bangbang and Fan, Tianxing and Yang, Zesong and Bao, Hujun and Zhang, Guofeng and Cui, Zhaopeng},
    booktitle={The IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR)},
    year={2023}
}

Acknowledgement

In this project we use parts of the implementations of the following works:

• nerf_pl by kwea123
• Splice by omerbt

We thank the respective authors for open sourcing their methods.