conformer icon indicating copy to clipboard operation
conformer copied to clipboard

Published 20 hours ago • verified publisher icon lucidrains

Not found LayerNorm in FeedForward module and Attention module

Open CSLujunyu opened this issue 3 years ago • 1 comments

As the paper described, LayerNorm layer are added to FeedForward module and Attention module. But I can't find it in this project.

image image

Is it something wrong?

CSLujunyu avatar May 31 '21 08:05 CSLujunyu

They are here:

https://github.com/lucidrains/conformer/blob/0e91f0323089cc00fbde3e112a984d94d0af09b8/conformer/conformer.py#L56-L64

and

https://github.com/lucidrains/conformer/blob/0e91f0323089cc00fbde3e112a984d94d0af09b8/conformer/conformer.py#L189-L191

enhuiz avatar Oct 28 '21 07:10 enhuiz