BERT-pytorch icon indicating copy to clipboard operation
BERT-pytorch copied to clipboard

Fix non-matching tensor with odd embedding size.

Open zmy opened this issue 5 years ago • 0 comments

If d_model integer parameter is odd rather than even, then tensor size is not matching (by a difference of 1) on the left and right sides of the assignment:

pe[:, 1::2] = torch.cos(position * div_term)

This pull request fixes the problem.

zmy avatar Jan 26 '19 15:01 zmy