BERT-pytorch

BERT-pytorch copied to clipboard

PositionalEmbedding

3

The position embedding in the BERT is not the same as in the transformer. Why not use the form in bert?

fgaoyang

if I want to do NER,so What should I do with your code

DeShuiYu

The base BERT model in https://arxiv.org/pdf/1810.04805.pdf uses 768 hidden features, 12 layers, 12 heads (which are also the defaults in `bert.py`), while the default configuration in the argparser of `__main__.py`...

bertmaher

I'm relatively new to pytorch and BERT, but I figured out how to pretrain the BERT model on my dataset. I want to use the model to predict the mask(s)...

sfaux

pred_loss decrease fast while avg_acc stay at 50%

53

I try to run the code on a small dataset and I find that pred_loss decrease fast while avg_acc stay at 50%. It is strange to me since decrease in...

jiqiujia

Hi! I am new to pytorch. So, could you provide a small example to illustrate how to use this pre-trained model to load, inference and finetune? Thx

dongrixinyu

How to Output Embedded Sentence Vector?

2

rhypowang

Predicting 'is_next' seems incorrect. Since the 'is_next' label value consists of 0 and 1, ignore_index should be 'None'.

watseob

How to Output Embedded Word Vector

6

I want to output the word vector

enze5088

0GSC0-0