BBPE issues

Results 2 BBPE issues

Sort by recently updated

运行train.py报错

Traceback (most recent call last): File "/home/quiana/work/python-toturial/Tokenization/BBPE/train.py", line 11, in BBPETokenizer.train_tokenizer(data, vocab_size, vocab_outfile=vocab_outfile, merges_outfile=merges_outfile) File "/home/quiana/work/python-toturial/Tokenization/BBPE/bbpe.py", line 170, in train_tokenizer most_common_pair = pair_freq.most_common(1)[0][0] ~~~~~~~~~~~~~~~~~~~~~~~~^^^ IndexError: list index out of range

lucheng07082221