MoGe icon indicating copy to clipboard operation
MoGe copied to clipboard

Wrong depth results ?

Open juntaosun opened this issue 1 year ago • 2 comments

(1)After testing some images, the resulting depth seems to be quite wrong. What could be causing this?

(2)Also, compared to Depth-Anything-V2, MoGe's output seems to be of lower resolution?


Image

Image

Image

juntaosun avatar Dec 07 '24 11:12 juntaosun

Hi, thank you for trying out our model and providing feedback.

The holes in the visualized depth map are due to MoGe's mask prediction. MoGe includes an additional head to predict a mask for regions of solid geometry. It identifies defocused, sky, and pure-colored background regions as "invalid," assigning infinite depth to these areas. In particularly challenging cases, this behavior can be more pronounced.

Currently, MoGe's depth prediction is not as visually sharp as the quality offered by Depth Anything V2. We've recently identified some issues in our training data that have led to blurriness and missing details in particular patterns, and we're actively working on retraining the model with corrected data.

Note: Since MoGe prioritizes 3D geometry accuracy over depth map visualization (aiming for high-quality 3D point clouds rather than visually appealing colorized depth maps), we recommend using Depth Anything V2 for tasks like depth controlnet, where visual sharpness is more critical than geometric accuracy.

Thank you for your understanding and support!

EasternJournalist avatar Dec 08 '24 06:12 EasternJournalist

Thank you very much, looking forward to the new model.

juntaosun avatar Dec 08 '24 09:12 juntaosun