Testing LTC

[henrytwo]

Hey @silvasean, we're going to start writing tests for the LTC components of Torch MLIR soon.

Here's what we were thinking:

• e2e test for the example LTC backend implementation, which will ensure lowering from PyTorch to MLIR works correctly
• upstream the shape inference functions from this PR to PyTorch itself, which will come with its own set of tests (though that won't land in torch-mlir)
• potentially having a test for each supported op?

What are your thoughts on having full coverage for all ops? There are quite a few of them, so we wouldn't be doing this all at once (and would probably require some community help). It's not strictly necessary, but would be nice to have.

cc: @antoniojkim @ke1337

[powderluv]

@henrytwo do we have a list of Ops somewhere ? Which model do you plan to use for the e2e test ? Something like BERT-L should already have all ops but happy to help add other ops for other models to test via LTC.

[antoniojkim]

do we have a list of Ops somewhere ?

A full list of ops is autogenerated from GeneratedTorchOps.td. Basically any op on there that can be codegen'd is included. There are a select few ops that are not supported and are blacklisted for the moment.

Something like BERT-L should already have all ops

The lowering via torch.jit is the same so I'd imagine those tests are a good place to start. However, I think we might want some LTC specific tests. Ones that specifically make sure that ops can be lowered correctly via LTC. This probably means that there will likely be some overlap

[powderluv]

yes yes. good to have LTC specific tests. Once you have a full training graph we can also add it to our downstream tests.

[silvasean]

The current e2e testing is enough to close this issue, I think. I don't think per-op non-e2e tests add much value here.