gsplat
gsplat copied to clipboard
Benchmark script (gsplat evaluation metrics)
Note, this PR does not have to be merged. It can stay as a PR or if people find it useful, it can be merged.
As mentioned in #323, here is a benchmark script to run various configurations of the gsplat library. This script was used to reproduce the metrics shown in https://github.com/nerfstudio-project/gsplat/commit/1cc3d22d2e5455988b06e1fc8b7c00e3ea5f32f2.
This script is inspired by @niujinshuchong work.
This script is usable for multi-gpu batch runs.
It makes a queue of jobs of the form [(config_0, scene_0), (config_0, scene_1), ... (config_0, scene_n), (config_1, scene_0), ...] and available GPUs take a config from the queue and processes it.
Configs can be defined easily. E.g.
baseline_config = BenchmarkConfig()
mcmc_config = BenchmarkConfig(
trainer="simple_trainer_mcmc.py",
result_dir="results/mcmc",
)
# Configs to run in batch
configs_to_run = [baseline_config, mcmc_config]