Machine-Learning-Collection icon indicating copy to clipboard operation
Machine-Learning-Collection copied to clipboard

Published 20 hours ago • verified publisher icon aladdinpersson

attention = torch.softmax(energy / (self.embed_size ** (1 / 2)), dim=3)

Open cwq159 opened this issue 4 years ago • 0 comments

should be attention = torch.softmax(energy / (self.head_dim ** (1 / 2)), dim=3)

cwq159 avatar Jul 01 '21 09:07 cwq159