Check Grad state before backprop AND preserve RNG state

[anijain2305]

@Chillee

One thing I am little skeptical about this is that - I could not come up with a handwritten testcase where the output requires_grad is True and grad_state is False.

But, this scenario comes up in TorchDynamo. Not sure if something is wrong with TorchDynamo here.

[Chillee]

Can we just check if grad is enabled before we call autograd.grad?

I do wonder whether we can have a more minimal repro though - will take a closer look tomorrow.

[anijain2305]

Can we just check if grad is enabled before we call autograd.grad?

So silly of me. Yeah, fixed this.

I do wonder whether we can have a more minimal repro though - will take a closer look tomorrow.

Yes, I did not try super hard, but I could not come up in few min.