BERT-pytorch
BERT-pytorch copied to clipboard
codertimo
Google AI 2018 BERT pytorch implementation
The way the trainer is set up the `iteration` that is used for train and test is similar except when train step is run the backwards propagation occurs. But one...
# trainer/pretrain.py ```python class BERTTrainer: def __init__(self, ...): ... # Using Negative Log Likelihood Loss function for predicting the masked_token self.criterion = nn.NLLLoss(ignore_index=0) ... ``` I cannot understand why `ignore...
I try to run according to md tutorial but 
such as if set d_model = 29, max_len = 10000 RuntimeError: The expanded size of the tensor (14) must match the existing size (15) at non-singleton dimension 1. Target sizes:...
For learning purposes, I added `example.ipynb`, which is a Google Colab Notebook that works right out of the box. I have also included an example data file that addresses #59...
I noticed that their is a pooler layer in original BERT that seems not present here. Did I miss something?
Hello, I have tried to run _bert_ with _--with_cuda False_, but the model keeps running "forward" function on cuda. These are my command line and the error message I got....
in the dataset.py line 31, it seem to report error random.randint() needs two positional argument.
