pytorch-transformer icon indicating copy to clipboard operation
pytorch-transformer copied to clipboard

Published 20 hours ago • verified publisher icon hkproj

Regarding add and norm block

Open DevasishY opened this issue 1 year ago • 1 comments

The provided code is x+sublayer(self.norm(x)) in model.py residual connection function but in paper it mentioned add and norm, that does mean self.norm(x+sublayer(x)). please clarify the same.

DevasishY avatar Jul 26 '24 17:07 DevasishY

Yes，The Paper is differ from Umar's code Umar comfirmed that in his video

SeekPoint avatar Dec 01 '24 17:12 SeekPoint