[testing] Insufficient coverage in test suite

[kshitij12345]

In functorch test suite, we use sample_inputs to get samples from an OpInfo. The problem is that sample_inputs may or may not cover all the case/overloads for an operator. I think we should use reference_inputs which super set of sample_inputs and more comprehensive. (Though this will increase the test times).

Switching sample_inputs to reference_inputs leads to bunch of failure for test_op_has_batch_rule including the ones mentioned in https://github.com/pytorch/functorch/issues/1080 https://github.com/pytorch/functorch/issues/1069

Refer to https://github.com/pytorch/pytorch/pull/91355 for failures.

cc: @zou3519

[zou3519]

Good catch!

Do we have a sense of how much this increases testing time by?

[kshitij12345]

Will confirm this in a day or two.

[kshitij12345]

Timings are posted here : https://github.com/pytorch/pytorch/pull/91355#issue-1509073244

[lezcano]

This one should probably be closed

[kshitij12345]

I want to check other tests in functorch/test_ops.py like test_vmapvjp and friends. So keeping it open till then.