Error: cannot import name 'Unigram' from fast_bert.data_cls import BertDataBunch

[zrajabi]

Another weird error once I was running in colab:

in () ----> 1 from fast_bert.data_cls import BertDataBunch 2 from fast_bert.learner_cls import BertLearner 3 from fast_bert.metrics import accuracy 4 import logging 5 import torch

7 frames /usr/local/lib/python3.6/dist-packages/transformers/convert_slow_tokenizer.py in () 22 23 from tokenizers import Tokenizer, decoders, normalizers, pre_tokenizers, processors ---> 24 from tokenizers.models import BPE, Unigram, WordPiece 25 26 # from transformers.tokenization_openai import OpenAIGPTTokenizer

ImportError: cannot import name 'Unigram'

---

NOTE: If your import is failing due to a missing package, you can manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the "Open Examples" button below.

[asnota]

As proposed in a similar issue https://github.com/huggingface/transformers/issues/7806, upgrade tokenizers. For Colab this worked for me: %pip install --upgrade tokenizers

[andrew-miao]

Make sure the version of tokenizers >= 0.8.1rc1. I also face this issue. You can create a new notebook and install tokenizers %pip install tokenizers before install fast-bert.