albanD

Not sure how autograd hooks interact with higher order gradients done via functorch cc @zou3519 do you have example of that already?

This would need to be part of a bigger plan to move things like `combine_state_for_ensemble` as well? Also this seems to be very vmap specific?

> Are you suggesting that we should put the nn.Ensemble API into functorch because it is vmap specific? Not necessarily but it does sound much "higher level" than things currently...

Just make https://github.com/pytorch/pytorch/blob/e4ea751810bd1b27a105ac43ce2c8c84fabc1167/c10/core/TensorImpl.h#L1084 return `false` for BatchedTensor and this will go away! :)

There is special logic in all of autograd for this :) It basically will replace all the places where we would usually call `as_strided()` to now call the original view...

The difference is noticeable for very small ops but not a dealbreaker either: ```python In [12]: a = torch.view_as_real(torch.rand(2, dtype=torch.complex64, requires_grad=True).clone()) In [13]: b = torch.rand(4, requires_grad=True).clone().view(2, 2) In [14]:...

> Isn't it roughly that its the same type of tensor (and memory format?) as the output tensor? Same shape / dtype / layout / device (checks are here https://github.com/pytorch/pytorch/blob/5fbec86faef07d66ab696bc4c4edbaf6259a2189/torch/csrc/autograd/engine.cpp#L694)....

Hey! https://dev-discuss.pytorch.org/t/what-and-why-is-torch-dispatch/557 should be a good intro

FYI @rohan-varma

Ho yes the default one will never work. You should pass `use_reentrant=False` to get the version that doesn't use custom Function.