tlc-pack
[Op] Implement several high-priority tensor ops: dense, conv2d, max_pool2d, relu, softmax, flatten
co-authored by: @jinhongyii
This PR implements several high-priority relax operators: dense, conv2d, max_pool2d, relu, softmax, flatten.
To support these new operators, LowerWithRelayOpStrategy pass has been extended to consider attribute.
Caveat: this PR only tests a very simple case. We would like to conduct more exhaustive testing before the official usage.
cc. @YuchenJin @slyubomirsky
For testing, there is an easy route available for gathering more test cases: You can port over some of the Relay tests for these ops, of which there are quite a few.
For testing, there is an easy route available for gathering more test cases: You can port over some of the Relay tests for these ops, of which there are quite a few.
@slyubomirsky, it seems like Relay does not have comprehensive op testing neither? Seems pretty much similar with what this PR has. https://github.com/tlc-pack/relax/blob/relax/tests/python/relay/test_op_level1.py Are there more testing you are aware of?
it seems like Relay does not have comprehensive op testing neither? Seems pretty much similar with what this PR has.
It's not perfect, but it's a start haha. Not every op has very comprehensive testing. The way to get "truly" comprehensive testing is fuzzing with a reference implementation, but that is a bit of trouble to implement...
agree on the runtimedep shape part, perhaps we can do one bulk change after this PR