Ex4DGS

Ex4DGS copied to clipboard

Fully Explicit Dynamic Gaussian Splatting

Junoh Lee · ChangYeon Won · Hyunjun Jung · Inhwan Bae · Hae-Gon Jeon
NeurIPS 2024

Project Page NeurIPS Paper Arxiv Paper Source Code Related Works

A novel view synthesis result of Ex4DGS.

Summary: 4D Gaussian Splatting with static & dynamic separation using an incrementally extensible, keyframe-based model

Contents

1. Setup
2. Preprocess Datasets
3. Training
4. Evaluation
5. Pretrained models

Setup

Environment Setup

Clone the source code of this repo.

git clone https://github.com/juno181/Ex4DGS.git
cd Ex4DGS

Installation through pip is recommended. First, set up your Python environment:

conda create -n Ex4DGS python=3.9
conda activate Ex4DGS

Make sure to install CUDA and PyTorch versions that match your CUDA environment. We've tested on RTX 4090 GPU with PyTorch version 2.1.2. Please refer https://pytorch.org/ for further information.

pip install torch

The remaining packages can be installed with:

pip install --upgrade setuptools cython wheel
pip install -r requirements.txt

Preprocess Datasets

For dataset preprocessing, we follow STG.

Neural 3D Video Dataset

First, download the dataset from here. You will need colmap environment for preprocess. To setup dataset preprocessing environment, run scrips:

./scripts/env_setup.sh

To preprocess dataset, run script:

./scripts/preprocess_all_n3v.sh <path to dataset>

Technicolor dataset

Download the dataset from here. To setup dataset preprocessing environment, run scrips:

./scripts/preprocess_all_techni.sh <path to dataset>

Please refer STG for further information.

Training

Run command:

python train.py --config configs/<some config name>.json --model_path <some output folder>  --source_path <path to dataset>

Evaluation

Run command:

python render.py --model_path <path to trained model>  --source_path <path to dataset> --skip_train --iteration <trained iter>

Pretrained models

We provide pretrained models in release.

📖 Citation

🛍️ Ex4DGS (NeurIPS'24) 🛍️ |

@inproceedings{lee2024ex4dgs,
  title={Fully Explicit Dynamic Guassian Splatting},
  author={Lee, Junoh and Won, ChangYeon and Jung, Hyunjun and Bae, Inhwan and Jeon, Hae-Gon},
  booktitle={Proceedings of the Neural Information Processing Systems},
  year={2024}
}