BERT-pytorch icon indicating copy to clipboard operation
BERT-pytorch copied to clipboard

IndexError

Open LemonQC opened this issue 3 years ago • 6 comments

I try to run according to md tutorial but image

LemonQC avatar Mar 05 '21 14:03 LemonQC

same issue

Wushengyao avatar Jun 23 '21 03:06 Wushengyao

hey bro,do you have solutions?

zxhjames avatar Aug 17 '21 01:08 zxhjames

becauese there are some problems abouot spilt('\t').

syzhy113 avatar Feb 07 '22 14:02 syzhy113

same issue, any solutions?

shazifan avatar Jun 29 '22 13:06 shazifan

check your corpus, it might not follow this rule in some lines: 'text \t text'

cheng940929 avatar Jun 30 '22 08:06 cheng940929

It likes the parser for '\t' works bad..

if you install bert_pytorch by pip, edit /opt/conda/lib/python3.8/site-packages/bert_pytorch/dataset/dataset.py line 23.

change

self.lines = [line[:-1].split("\t")
                              for line in tqdm.tqdm(f, desc="Loading Dataset", total=corpus_lines)]

to

self.lines = [line[:-1].split("\\t")
                              for line in tqdm.tqdm(f, desc="Loading Dataset", total=corpus_lines)]

Chasingdreams6 avatar Jan 12 '24 02:01 Chasingdreams6