tpp-mlir icon indicating copy to clipboard operation
tpp-mlir copied to clipboard

Generalize `conv-simplify` for matmuls, add residual optimization

Open rengolin opened this issue 1 year ago • 0 comments

The pass conv-simplify moves the bias add to the tensor initialization of a convolution if it's a zero-splat. This is common in matmul networks too, so we should make that a more generic pass. We should then rename it to linalg-simplify or something.

Another optimization is to move the bias add before the conv in a residual layer, so that we can fuse the residual conv with the bias add before the bottleneck conv, which then fuses with the final ReLU.

This needs to be in the default pipeline and have integration tests added.

@chelini @adam-smnk

rengolin avatar Mar 03 '23 17:03 rengolin