BBPE icon indicating copy to clipboard operation
BBPE copied to clipboard

运行train.py报错

Open lucheng07082221 opened this issue 1 year ago • 0 comments

Traceback (most recent call last): File "/home/quiana/work/python-toturial/Tokenization/BBPE/train.py", line 11, in BBPETokenizer.train_tokenizer(data, vocab_size, vocab_outfile=vocab_outfile, merges_outfile=merges_outfile) File "/home/quiana/work/python-toturial/Tokenization/BBPE/bbpe.py", line 170, in train_tokenizer most_common_pair = pair_freq.most_common(1)[0][0] ~~~~~~~~~~~~~~~~~~~~~~~~^^^ IndexError: list index out of range

lucheng07082221 avatar Oct 10 '24 08:10 lucheng07082221