3DEnhancer icon indicating copy to clipboard operation
3DEnhancer copied to clipboard

3DEnhancer: Consistent Multi-View Diffusion for 3D Enhancement

3DEnhancer: Consistent Multi-View Diffusion for 3D Enhancement

3DEnhancer employs a multi-view diffusion model to enhance multi-view images, thus improving 3D models.

pipeline :open_book: For more visual results, go checkout our project page

Introducing 3DEnhancer
pipeline

Despite advances in neural rendering, due to the scarcity of high-quality 3D datasets and the inherent limitations of multi-view diffusion models, view synthesis and 3D model generation are restricted to low resolutions with suboptimal multi-view consistency. In this study, we present a novel 3D enhancement pipeline, dubbed 3DEnhancer, which employs a multi-view latent diffusion model to enhance coarse 3D inputs while preserving multi-view consistency. Our method includes a pose-aware encoder and a diffusion-based denoiser to refine low-quality multi-view images, along with data augmentation and a multi-view attention module with epipolar aggregation to maintain consistent, high-quality 3D outputs across views. Unlike existing video-based approaches, our model supports seamless multi-view enhancement with improved coherence across diverse viewing angles. Extensive evaluations show that 3DEnhancer significantly outperforms existing methods, boosting both multi-view enhancement and per-instance 3D optimization tasks.

:fire: News

  • [2024/12/25] Our paper and project page are now live. Merry Christmas!

:calendar: TODO

  • [x] Release paper and project page.
  • [ ] Release code (coming soon!).
  • [ ] Release Gradio demo.

:page_with_curl: License

This project is licensed under NTU S-Lab License 1.0. Redistribution and use should follow this license.

:pencil: Citation

If you find our code or paper helps, please consider citing:

@article{luo20243denhancer,
    title={3DEnhancer: Consistent Multi-View Diffusion for 3D Enhancement}, 
    author={Yihang Luo and Shangchen Zhou and Yushi Lan and Xingang Pan and Chen Change Loy},
    booktitle={arXiv preprint arXiv:2412.18565}
    year={2024},
}

:mailbox: Contact

If you have any questions, please feel free to reach us at [email protected].