Correct DIB-R texture optimization behavior?

[aluo-x]

As a proof of concept, I rendered an object in blender, and re-used the same object position and camera parameters to render an image in Blender. I tried to regress a texture in both Lambertian and VertexColor modes directly (without any networks).

It seems like in VertexColor mode, the colors are jumping in large steps, and in the Lambertian mode, there are stuck pixels despite perfect object alignment. This is slightly concerning, as an experiment using neural mesh, the colors change very smoothly.

GIFs of the optimization using DIB-R: https://imgur.com/a/xfevKGO

@SteveJunGao @TommyX12 @wenzhengchen is this the correct behavior?

[aluo-x]

Actually fixed the bug. DIB-R was missing a check, fixed it in my private version.

[krrish94]

@aluo-x Could you please point us to the bug, if it's still there?

[aluo-x]

I believe it had to do with the gradients causing the texture values to go negative. I recall doing the following:

1. Sigmoid applied to vertex color (significantly slowed convergence)
2. Hard clip (textures got stuck)
3. Strong L2 penalty for negative (less than -eps textures) (worked)
4. Fake clamp (suggested by nikhilaravi for an unrelated problem), don't really recall if this worked or not

class fakeclamp(torch.autograd.Function):
    @staticmethod
    def forward(
        ctx,
        input_tensor,
        min=0.0,
        max=1.0,
    ):
        ctx.save_for_backward(input_tensor)
        return torch.clamp(input_tensor, min=min, max=max)

    @staticmethod
    def backward(
        ctx, grad_output_tensor
    ):
        input_tensor = ctx.saved_tensors[0]
        grad_input_tensor = input_tensor.new_ones(input_tensor.shape) * grad_output_tensor
        return grad_input_tensor, None, None

I've switched over my code base to pytorch3d, and its been a few months since I've used DIB-R, apologies.

[Caenorst]

Closing for inactivity