stable-dreamfusion icon indicating copy to clipboard operation
stable-dreamfusion copied to clipboard

New Paper: ProlificDreamer - several ideas to improve detail/quality

Open mhale1 opened this issue 1 year ago • 3 comments

Maybe you've already seen this, but a new paper just came out. They call it VDS optimization instead of SDS. Fixes saturated colors, lack of variety, and smoothed details. Requires an extra LoRA. But even without Zero123 they are getting some very good results bumping up to 512 resolution. Check out their pineapple. https://ml.cs.tsinghua.edu.cn/prolificdreamer/

https://github.com/ashawkey/stable-dreamfusion/assets/5043964/af385718-5509-4f87-86f7-9ff8d57f0f4d

mhale1 avatar May 26 '23 11:05 mhale1

This works seems very exciting, is there a plan to incorporate the techinques into Stable-Dreamfusion projects?

pkulwj1994 avatar May 30 '23 13:05 pkulwj1994

https://arxiv.org/abs/2305.18766

And now another loss function comes to light :)

Seems very interesting and it's able to skip the refinement stage.

jclarkk avatar May 31 '23 11:05 jclarkk

https://arxiv.org/abs/2305.18766

And now another loss function comes to light :)

Seems very interesting and it's able to skip the refinement stage.

I implement BGT+ loss and test it on the dmtet stage, but it did not show improvement. If kiui has interest to implement it, I would like to compare the code and know any details are ignored or misunderstood

mdarhdarz avatar Jun 01 '23 05:06 mdarhdarz