MVDream

[snicolast]

Hi, thanks for the MVDream addition! txt-2-3d looks better now in terms of geometry consistency (waiting for consistent1to3 technique tho!).

Just a question.. why the final textures (after refinement stage) always look like this (a mix of green, pink, etc)? These results are different from the original paper. Any ideas? Thanks!

[ashawkey]

@snicolast Hi, I have obeserved this too in the second stage. I still haven't figured out why, but you may try to decrease iters_refine to alleviate it: https://github.com/dreamgaussian/dreamgaussian/blob/main/configs/text_mv.yaml#L30C6-L30C6

[snicolast]

Hi @ashawkey, thanks! I gave it a shot, but I didn't notice a significant improvement. I even attempted the opposite approach, refining it with about 50 steps. Here's a comparison of the images. I can't really discern a substantial difference among the three, though perhaps the 50-step version appears slightly more "solid." However, in terms of texture, they all seem quite similar (prompt is "a tractor").

Offtopic:

btw, there is an error if I try 500 steps for the coarse stage:

File "...\gs_renderer.py", line 237, in extract_fields mn, mx = xyzs.amin(0), xyzs.amax(0) IndexError: amin(): Expected reduction dim 0 to have non-zero size.

[ashawkey]

These tractors seem good to me? You may check the first stage model too. If it is already green, then the second stage may not change the color. That error means the Gaussians diverged and failed to form an object, it may happen if parameters are improper.

[Texaser]

Hi @ashawkey, thanks! I also tried it, but it seems that the result is a little bit unsatisfactory. The shape of this tiger is a little bit strange (like a cuboid) and the color is not like a real tiger. Even the result is not that good as your repo stable dreamfusion. The prompt is "a tiger cub".

https://github.com/dreamgaussian/dreamgaussian/assets/50570271/d7990bc1-889b-4c1d-8700-87986a6f47fa

Could you please give me some intuition about why the result is not that perfect? and how can we improve the quality?

[Asubayo]

Hi @ashawkey, thanks! I gave it a shot, but I didn't notice a significant improvement. I even attempted the opposite approach, refining it with about 50 steps. Here's a comparison of the images. I can't really discern a substantial difference among the three, though perhaps the 50-step version appears slightly more "solid." However, in terms of texture, they all seem quite similar (prompt is "a tractor").

Offtopic:

btw, there is an error if I try 500 steps for the coarse stage:

File "...\gs_renderer.py", line 237, in extract_fields mn, mx = xyzs.amin(0), xyzs.amax(0) IndexError: amin(): Expected reduction dim 0 to have non-zero size.

Hello ! May I ask you what was your prompt ? Only "a tractor" ? Because I get a total different result ...

[Asubayo]

All good, I am getting the same result by switching on mvdream

[mdarhdarz]

I think gradient clipping on image space can help a lot although no mvdream only experiments and stage2 refine stage. from left to right: zero123, zero123+mvdream, zero123+mvdream+pgc(https://fudan-zvg.github.io/PGC-3D/) background is zeronvs guidance

https://github.com/dreamgaussian/dreamgaussian/assets/84657631/b09563e6-fc67-4fe2-a3d4-d1b52469dc70

[qixuanwang-233]

Hello, I encountered an error while using mvdream: RuntimeError: The shape of the 2D attn_mask is torch.Size([77, 77]), but should be (1, 1). Did you encounter this error and how did you solve it?