gpt-2-simple icon indicating copy to clipboard operation
gpt-2-simple copied to clipboard

why gpt encoder norm is before mlp while the original transformer is mlp before norm?

Open githubutilities opened this issue 3 years ago • 0 comments

https://github.com/minimaxir/gpt-2-simple/blob/master/gpt_2_simple/src/model.py#L158 Original paper: https://arxiv.org/pdf/1706.03762.pdf

githubutilities avatar Mar 29 '21 02:03 githubutilities