BERT-pytorch
BERT-pytorch copied to clipboard
codertimo
Google AI 2018 BERT pytorch implementation
I mean,not just use BERT model to predict answers,also train it.
Well all of you guys know, it's nearly impossible to train from the scratch, because of lack of computation power. So I'm going to implement the transfer code for making...
https://github.com/codertimo/BERT-pytorch/blob/d10dc4f9d5a6f2ca74380f62039526eb7277c671/bert_pytorch/model/attention/multi_head.py#L15 Looks that **self.d_k = d_model // h ---> embed size 768 dividing number of heads 12 = 64** ``` self.d_k = d_model // h # 64 self.h = h...
predict next sentence only? it also can process SQA assignments?
Could you please give some example code to load in the pre-trained model(i.e. bert.model.ep0 files)? The code might take me a while to understand so I really appreciate it if...
my model **-hs 60 -l 3 -a 3 -s 26 -b 5 -e 10 -w 4 --with_cuda True --log_freq 20 --on_memory False --lr 1e-3 --adam_weight_decay 0.0 --adam_beta1 0.9 --adam_beta2 0.999**...
The line referencing `self.load_vectors` https://github.com/codertimo/BERT-pytorch/blob/7efd2b5a631f18ebc83cd16886b8c6ee77a40750/bert_pytorch/dataset/vocab.py#L63 will fail as this method doesn't appear to exist (on quick search of the repo). Apologies if I've misunderstood something obvious...
I'm wondering if there's any workaround for an error when trying to convert BERT-pytorch to ONNX. The problem occurs when trying to trace through the TransformerBlock. I'm wondering if there's...
I am using for next sent. gen so while it has stored model in .ep* format but how to run my test dataset using those models. Thanks
