Use OpInfos for operator tests

[ezyang]

🐛 Describe the bug

Defining OpInfos for torchvision's operators would make it possible to run stock PyTorch correctness tests (that are generic over operators) in torchvision. For example, I want to add meta function support to torchvision operators but my preference is to reuse the testing infrastructure in PyTorch.

cc @pmeier @mruberry I don't know if it is actually supported to run tests in PyTorch with extra opinfos, doesn't feel like it.

Versions

master

[mruberry]

We don't support this today but I don't think we're far off. We could just extend op_db with torchvision OpInfos when torchvision is available, for example, or we could require tests opt into ops_and_torchvision_ops like we do with Python Reference implementations currently.

[justinchuby]

Curious if there is any updates on this?

[NicolasHug]

We now rely on optests.opcheck to check the correctness of the custom ops w.r.t to torch.compile().

https://github.com/pytorch/vision/blob/1de7a74a8b93483f1703eef0b306e0ec68e0cd9d/test/test_ops.py#L704

I'm not sure if that's exactly what "opinfos" are about though, as I'm not familiar with what happens within the torch core test base. But relying on opcheck addresses the original need above regarding testing meta functions.