Anushagudipati

getting issues with tokenizer

5

unable load Tokenizer using AutoTokenizer.from_pretrained() errors: tokenizer = AutoTokenizer.from_pretrained(model_id) File "/home/ubuntu/venv/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 862, in from_pretrained return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs) File "/home/ubuntu/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2089, in from_pretrained return cls._from_pretrained( File "/home/ubuntu/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line...