Problems on other NeRF models

[C0SM00]

I tried to reproduce your gradient supervision method on the Tensorf model, but the calculated rgb-to-coordinate gradient is not in the same order of magnitude as the target gradient. The former is around 1e-6, and the latter is around 1e-1. Excuse me. Is there a problem with reproduction?

[wtyuan96]

Thank you for your interest in our work.

We have not test our method with TensoRF, but you can try from the following aspects:

1. check the the input coordinate of the function diff_operators.gradient, the coordinate should not be normalized, i.e. the coordinate should be the same as when calculating the target gradient.
2. calculate the rgb-to-coordinate gradient after the TensoRF model optimization (without supervision on derivatives) is complete, not when the model is initialized.
3. Some other factors that may have an impact: use LLFF dataset, remove view direction dependency.

Good luck!

[C0SM00]

I ran your original code with the command line python train.py --config configs/fern/val_der/sine.txt, but why didn't I observe a drop in der_loss?

[wtyuan96]

Maybe you can check your reproduction as I said first. By the way, TinyNeRF and TensoRF are different implementations of INRs, i.e. frequency encoding and parametric encoding (from Instant-NGP), the activation function and the approximation capability of derivative may also have an impact.