transformer icon indicating copy to clipboard operation
transformer copied to clipboard

Published 20 hours ago • verified publisher icon hyunwoongko

shaollow copy

Open xxoospring opened this issue 1 year ago • 1 comments

this is a shaollow copy, makes the "_x" and "x" totally the same one? https://github.com/hyunwoongko/transformer/blob/0e5ce57589d7307cf76b53241cc523841ff67655/models/blocks/encoder_layer.py#L27

xxoospring avatar Nov 08 '23 11:11 xxoospring

It does not matter since the code does not modify x or _x in-place. In the next row, x is overwritten by the output of the attention

OskarLiew avatar Nov 18 '23 14:11 OskarLiew