nvdiffrec Normal map for training

Hi! I am a bit confused in the training part using meshes. I am trying to use the normal map as a parameter for supervision for guiding the loss function. My goal would be to build a dataset from .obj files and use the renderer for creating the images, as well as their normal map. You mentioned in #85 that normal map was directly derived from the .obj file for training, but I am wondering how the supervision could work if the normal map derived from the renderer is smoothed but the one from the .obj file is fine. If I am rendering the normal map of a cube I am getting something like that: normal_map When I would normally expect the normal map render to look like that: cube_normal

Not sure if I am missing something...

May 01 '24 12:05 Kev1MSL

Hello @Kev1MSL ,

I'm not sure I understand the issue. When loading a reference mesh from an .obj file, we respect the normals in the file if present: https://github.com/NVlabs/nvdiffrec/blob/main/render/obj.py#L115

For the mesh we optimize, we run auto_normals in each training step: https://github.com/NVlabs/nvdiffrec/blob/a3e73909a01887c8a135235ff860dd23a045cc1b/geometry/dlmesh.py#L42

May 02 '24 15:05 jmunkberg

Alright I think I understand better now, but I am not sure why you take the normal from the file, is it for supervision when you train your model?

May 03 '24 08:05 Kev1MSL