[QST] GEMM Epilogue Fusion: Element-wise Ops and Two-Tensor Element-wise Multiplication

[HanGuo97]

What is your question? Hi, I'd like to compute the following

D = f( matmul(A, B) ) * C

where f is an element-wise activation function, and C is a tensor of the same shape as the output D. In CUTLASS, it seems like only per-dimension scaling is supported in epilogue fusion, but not element-wise scaling between two tensors. In light of this, is such fusion profitable? If so, what are the best ways to implement this?

Thanks in advance for your time!