BERT-pytorch
BERT-pytorch copied to clipboard
embedding/position.py with RuntimeError if d_model is odd
such as if set d_model = 29, max_len = 10000 RuntimeError: The expanded size of the tensor (14) must match the existing size (15) at non-singleton dimension 1. Target sizes: [10000, 14]. Tensor sizes: [10000, 15] code in this line may be fixed (line 30): pe[:, 1::2] = torch.cos(position * div_term)
modify pe[:, 1::2] = torch.cos(position * div_term) to pe[:, 1::2] = torch.cos(position * div_term)[:, :d_model//2]
d_model should be a even number