zero-bubble-pipeline-parallelism icon indicating copy to clipboard operation
zero-bubble-pipeline-parallelism copied to clipboard

Published 20 hours ago • verified publisher icon sail-sg

[QUESTION] IS zero bubble pp support flash-atten?

Open qq1243196045 opened this issue 5 months ago • 1 comments

I SEE zero-bubble-pipeline-parallelism disabled FusdLayerNorm，Is it because of the fused op can not split backward of w and x？

qq1243196045 avatar Sep 05 '24 08:09 qq1243196045