DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

Transformer/fix layer norm

Open RezaYazdaniAminabadi opened this issue 2 years ago • 1 comments

This PR addresses https://github.com/microsoft/DeepSpeed/issues/581

RezaYazdaniAminabadi avatar Sep 27 '21 20:09 RezaYazdaniAminabadi

Can one of the admins verify this patch?

rocm-mici avatar Jun 09 '22 20:06 rocm-mici

Hi, was this issue fixed? LayerNorm combined with DeepSpeed FP16 still seems to be problematic.

FarzanT avatar Dec 16 '22 18:12 FarzanT