BERT-pytorch

BERT-pytorch copied to clipboard

Did you have tried to use ipdb to set breakpoint in __get__item__, which is a funtion from BERTDataset

1

moomooda

dataset / dataset.py have one erro?

1

" def get_random_line(self): if self.on_memory: self.lines[random.randrange(len(self.lines))][1] " This code is to get the incorrect next sentence（isNotNext : 0)， maybe it random get a lines it is (isnext:1)。

ndn-love

why language_model.py has different vectors

1

In language_model.py codes, class NextSentencePrediction and class MaskedLanguageModel has different input x in their forward function, In class NextSentencePrediction the input use "x[: 0] in "return self.softmax(self.linear(x[:, 0]))" but in...

zysNLP

the format of input

4

You mentioned that > NOTICE : Your corpus should be prepared with two sentences in one line with tab(\t) separator and gave an example: > Welcome to the \t the...

eveliao

Any one know where to get them? Thank you and thank you.

guotong1988

class BERTLM：self.next_sentence = NextSentencePrediction(self.bert.hidden) This error appeared when I ran: 【AttributeError: type object 'BERT' has no attribute 'hidden'】 How to deal with it? Thank you!

XueqiangF

```bash def get_random_line(self): ... return self.lines[random.randrange(len(self.lines))][1] ... ``` it should be changed to the following: ```bash def get_random_line(self,index): ... tmp = random.randrange(len(self.lines)) while(tmp == index): tmp = random.randrange(len(self.lines)) return self.lines[random.randrange(len(self.lines))][1]...

Emir-Liu

IndexError: list index out of range

13

ghost

transformer.py 中的forword方法调用的SublayerConnection类。实现残差链接和标准化的实现

1

sublayerout = layerNorm(x +sublayer(x)) **首先是残差链接然后是层标准化** 在你代码中：sublayer.py中 应该是 def forward(self, x, sublayer): "Apply residual connection to any sublayer with the same size." # return x + self.dropout(sublayer(self.norm(x))) return self.norm( x +...

dshwei

dshwei