RuntimeError: DifferentiableObject::backward_backward_input_impl: not implemented error (doutput)

[coder4869]

I met the following error, when modifing neus with tinycudann:

  File "exp_runner.py", line 397, in <module>
    runner.train()
  File "exp_runner.py", line 148, in train
    loss.backward()
  File "/home/nerf/.conda/envs/neus/lib/python3.7/site-packages/torch/_tensor.py", line 396, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/nerf/.conda/envs/neus/lib/python3.7/site-packages/torch/autograd/__init__.py", line 175, in backward
    allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass
  File "/home/nerf/.conda/envs/neus/lib/python3.7/site-packages/torch/autograd/function.py", line 253, in apply
    return user_fn(self, *args)
  File "/home/nerf/.conda/envs/neus/lib/python3.7/site-packages/tinycudann-1.6-py3.7-linux-x86_64.egg/tinycudann/modules.py", line 88, in backward
    doutput
RuntimeError: DifferentiableObject::backward_backward_input_impl: not implemented error

Here is the code:

# models/fields.py for NeuS
class SDFNetwork(nn.Module):
    def __init__():
       xxxx
        self.sdf_lin_net = tcnn.Network(
            n_input_dims    = (2*multires+1)*d_in,
            n_output_dims   = d_out, # out_dim, d_out
            network_config  = {
                "otype": "FullyFusedMLP", # FullyFusedMLP | CutlassMLP 
                "activation": "Softplus",
                "output_activation": "None",
                "n_neurons": d_hidden,
                "n_hidden_layers": n_layers,
            },
        )
    def forward(self, inputs):
        inputs = inputs * self.scale
        if self.embed_fn_fine is not None:
            inputs = self.embed_fn_fine(inputs)

        x = inputs
        y = self.sdf_lin_net(x)
        return y

# bindings/torch/tinycudnn/modules.py
	@staticmethod
	def backward(ctx, dinput_grad, dweight_grad):
		# NOTE: currently support:
		#       ✓   d(dL_dinput)_d(dL_doutput)  doutput_grad
		#       ✓   d(dL_dinput)_d(params)      weight_grad
		#       ✓   d(dL_dinput)_d(input)       input_grad
		#       x   d(dL_dparam)_d(...)
		input, params, doutput = ctx.saved_tensors
		# assert dweight_grad is None, "currently do not support 2nd-order gradients from gradient of grid"
		with torch.enable_grad():
			# NOTE: preserves requires_grad info (this function is in no_grad() context by default when invoking loss.backward())
			doutput = doutput * ctx.ctx_fwd.loss_scale
		with torch.no_grad():
			print("ctx.ctx_fwd.native_ctx =", ctx.ctx_fwd.native_ctx)
			print("input =", input.size(), input, input.dtype)
			print("params =", params.size(), params, params.dtype)
			print("dinput_grad =", dinput_grad.size(), dinput_grad, dinput_grad.dtype)
			print("doutput =", doutput.size(), doutput, doutput.dtype)
			doutput_grad, weight_grad, input_grad = ctx.ctx_fwd.native_tcnn_module.bwd_bwd_input(
				ctx.ctx_fwd.native_ctx,
				input,
				params,
				dinput_grad,
				doutput
			)
			# NOTE: be cautious when multiplying and dividing loss_scale
			#       doutput_grad uses dinput_grad
			#       weight_grad  uses dinput_grad * doutput
			#       input_grad   uses dinput_grad * doutput
			weight_grad = None if weight_grad is None else (weight_grad / ctx.ctx_fwd.loss_scale)
			input_grad = None if input_grad is None else (input_grad / ctx.ctx_fwd.loss_scale)

		# ctx_fwd,   doutput,      input,      params,      output
		return None, doutput_grad, input_grad, weight_grad, None

How to fix it ?

https://github.com/NVlabs/tiny-cuda-nn/issues/89 this does not work for me!

[Tom94]

The answer is the same as in #89 . You're trying to use tiny-cuda-nn's fully fused MLP, which doesn't support second derivatives through double backward, hence the crash. If you want second derivatives, you must only use the tiny-cuda-nn hash encoding and combine it with a regular PyTorch neural network. Cheers!

[coder4869]

Dear Thomas: Thanks for your reply, the crash is solved ! Best. From xuexing

Thomas Müller @.***> 于2022年8月10日周三 22:08写道：

The answer is the same as in #89 https://github.com/NVlabs/tiny-cuda-nn/issues/89 . You're trying to use tiny-cuda-nn's fully fused MLP, which doesn't support second derivatives through double backward, hence the crash. If you want second derivatives, you must only use the tiny-cuda-nn hash encoding and combine it with a regular PyTorch neural network. Cheers!

— Reply to this email directly, view it on GitHub https://github.com/NVlabs/tiny-cuda-nn/issues/131#issuecomment-1210726647, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHBWSPTV4O6FQYBF2FGRXDVYOZV5ANCNFSM56DQGKPQ . You are receiving this because you authored the thread.Message ID: @.***>