Where is the ground truth for SVBRDF?

[humphrey-newcoder]

Hi,

Thanks for you nice work! I have read your code, but I have several questions during reading.

1. Why should the input images be seperated into nbtargets+1 parts based on image width, and set one of them as input and the others as targets?
2. Why the rerender loss are computed between the svbrdf albedos and original image regions? From my understanding, question 1 is just to crop the images and then after preprocessing, resize them all to 256*256. The input will be sent to the network and output the svbrdf albedo. Then outputs(four svbrdf albedos) and targets(nbtargets parts of original images) will be both rerendered and the loss will be computed. However, in your paper, in Figure 5, you mentioned the ground truth of SVBRDF for rerendering, but I didn't find where the gt is loaded.
3. I want to visualize the rerendered results of the input, how should I get it?

Sorry for being novel in compter graphics and maybe the questions are a little bit stupid.. Hope for your kind response.

BR, Humphrey

[valentin-deschaintre]

Hello, thanks for your interest. To the best of my understanding of your question here are my answers:

1. This subdivision is used for the training data, it should not be applied when using the "eval" mode. The format of the training images is a large image with concatenated flash image + 4 target maps. Hence when loading them I split them.
2. GT is called target in this case. This is extracted as explain in 1
3. The algorithm generates a set of material maps, to render it you need to use a renderer of your choice which implements the right material model. You can get a rough preview using a cook torrance rendering algorithm, but the exact equation for this model are a bit different resulting in a different appearance (and described in the supplemental).

Best, Valentin

[humphrey-newcoder]

Hi, thanks for your kind response. Regarding Q3, is there any possibility that I can directly leverage your code to save the rerendered images? Since I have seen many different rerendered outputs like D_rendered, G_rendered, F_rendered, diffuse render, specular render and result in your code, which one can generate the visual effect in your paper?

Best, Humphrey

[valentin-deschaintre]

diffuse render and specular render will contain the renderings done for the loss computation, but the visualization of the paper where generated using Mitsuba.

[valentin-deschaintre]

Hi,

I can't see your comment on github somehow, but here is the intuition:

The renderings are never compared with the input image. The input image (a synthetic rendering ground truth material during training) is processed by the U-Net, which generates maps, which are then rendered in the loss and compared to renderings of the ground truth material under the same random conditions.

Best, Valentin

On Sat, 15 Jul 2023 at 03:54, Youxin XING @.***> wrote:

Hi, thanks for your work.

Reading the code, I found that the wo and wi are generated randomly. Then they are used to compute the material with SVBRDF maps. But the question is that the rendered material in this way may differ from the ground truth image reading from the dataset. Because the camera and light positions, light power are not matched in the rendering.

So is there any key insight that I didn't get? Looking forward to your reply!

Thank you very much~

— Reply to this email directly, view it on GitHub https://github.com/valentin-deschaintre/Single-Image-SVBRDF-Capture-rendering-loss/issues/3#issuecomment-1636635501, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAF5KACKJEKS4SDRBISINWDXQIA6FANCNFSM5FBXAACQ . You are receiving this because you commented.Message ID: <valentin-deschaintre/Single-Image-SVBRDF-Capture-rendering-loss/issues/3/1636635501 @github.com>