[LinalgExt] Remove attention tile and decompose

Open Groverkss opened this issue 1 year ago • 2 comments

Depends on: https://github.com/iree-org/iree/pull/17536

Jun 10 '24 12:06 Groverkss

Good to have this cleanup, but IIRC @harsh-nod mentioned there are cases where we found regular, non FA faster, so tileAndDecomposeAttention may still be useful there?

Jun 11 '24 00:06 raikonenfnu

Good to have this cleanup, but IIRC @harsh-nod mentioned there are cases where we found regular, non FA faster, so tileAndDecomposeAttention may still be useful there?

In those cases, we don't want to use flash attention decomposition. We need to implement AggregateOpInterface for attention op (we currently have it on online_attention op).

Jun 27 '24 00:06 Groverkss

Already landed as part of https://github.com/iree-org/iree/commit/dd3f2a392819d121fa5329a1c591be06ae9e887a

Nov 22 '24 17:11 Groverkss