DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

Triangular mask support for Transformer kernel

Open RezaYazdaniAminabadi opened this issue 3 years ago • 1 comments

Adding the new version of SoftMax for Transformer kernel to support Triangular mask used in GPT-based models.

This addresses https://github.com/microsoft/DeepSpeed/issues/828.

TODO: Add a unit test for guarding against this type of masking.

RezaYazdaniAminabadi avatar Apr 07 '21 17:04 RezaYazdaniAminabadi

Can one of the admins verify this patch?

rocm-mici avatar Jun 09 '22 20:06 rocm-mici