BERT-pytorch
BERT-pytorch copied to clipboard
Fix non-matching tensor with odd embedding size.
If d_model
integer parameter is odd rather than even, then tensor size is not matching (by a difference of 1) on the left and right sides of the assignment:
pe[:, 1::2] = torch.cos(position * div_term)
This pull request fixes the problem.