dreamfields-3D icon indicating copy to clipboard operation
dreamfields-3D copied to clipboard

Published 20 hours ago • verified publisher icon shengyu-meng

Hi when i train with a 2d image , the output model is not a replica of the image but acquires random features , any idea why that is?

Open Hamad-exarta opened this issue 2 years ago • 3 comments

Hamad-exarta avatar Nov 02 '22 05:11 Hamad-exarta

Is it that it acquires a label first and gets random weights from imagnet or resnet and interpolates accordingly?

Hamad-exarta avatar Nov 02 '22 05:11 Hamad-exarta

There is because the input image was not used as direct reference but embed as a feature vector by CLIP. Therefore, it is used as "visual language" for CLIP, which more emphasize on semantic meaning rather than pixel-wise.

shengyu-meng avatar Nov 03 '22 12:11 shengyu-meng

Thankyou for your clarification Is there way that we can provide it with an image or 3 views of an image and get an accurate mesh? Perhaps with diffusing feature as a text prompt to change the design with the text while maintaining the original 3d shape?

Hamad-exarta avatar Nov 04 '22 09:11 Hamad-exarta