Differentiability wrt vertex colors

[justanhduc]

Hi @pmh47. Thanks for sharing this wonderful project. I wonder whether Dirt is differentiable wrt vertex colors? I tried to learn some uv but I get None when trying to get the gradients. Thanks in advance!

[pmh47]

Yes, it is. If you define your model something like in those in the samples folder, it should be differentiable wrt vertex colors, or UVs and texture values. If you have a short code example where it fails, then paste it here and I can take a look.

[justanhduc]

Thanks. I found the problem. I have this code fragment

self.vertex_colors = tf.Variable(np.random.rand(6890, 3).astype('float32'), trainable=True)
vertex_colors = self.vertex_colors
renderer_color = RenderLayer(self.img_size, self.img_size, 3, vertex_colors, np.ones(3), self.faces,
                             [self.img_size, self.img_size], [self.img_size / 2., self.img_size / 2.],
                             name='render_color_layer')
self.rendered_color = [renderer_color(tf.stop_gradient(v)) for i, v in enumerate(self.vertices)]

in which the RenderLayer is Dirt. The gradient works fine for self.vertex_colors and my loss decreases. However, if I apply some transformation on self.vertex_colors, no matter how trivial it is, like vertex_colors = self.vertex_colors + 1, the loss won't change at all and self.vertex_colors doesn't change. Could you please let me know what potential problem there can be?

[pmh47]

There is nothing wrong with the code you pasted -- I can't see why the gradients would be zero here. Hence there may be a problem with your RenderLayer class itself, or elsewhere in your code. If you can provide a minimal runnable example that prints the incorrect gradient (and tell me which version of tensorflow & python you use), I can look at what's happening, but I suspect it's not a problem with DIRT itself.