Mao Yunfei comments

Repositories
Issues
Comments

Results 3 comments of


                                            Mao Yunfei

Use dense Adam by convert indexedslices to tensor in DIEN

I have move the changes of this PR to DIEN_TF2 folder. It will not influence the code in the original DIEN folder.

great work! When to release flash attention 4?

> Soon, 3-4 weeks Great!! Will backward kernel also be released?

Can we reduce the kernel latency for normal dispatch and combine when overlapping with gemm kernels through tuning the UNROLL_FACTOR?

Hi @yaning223 ，thanks for posting this interesting issue. I'd like to ask, in the case of overlap, did you **increase or decrease** the unroll factor to reduce the performance degradation...