error when using from indobenchmark

[fendiirfan]

System Info

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("indobenchmark/indobart-v2")

the error ValueError: Tokenizer class IndoNLGTokenizer does not exist or is not currently imported.

any help guys?

Who can help?

@ArthurZucker

Information

• [ ] The official example scripts
• [ ] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

Run

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("indobenchmark/indobart-v2")

Expected behavior

Can used

[ArthurZucker]

Hey! The tokenizer_class that was set in the configuration.json is wrong as the IndoNLGTokenizer does not exist in transformers. You should try to ask the other of the model on the community tab how to use it, or try to use:

from transformers import MBartTokenizer
tokenizer = MBartTokenizer.from_pretrained("indobenchmark/indobart-v2")

as it appears that the model is an MBartModel.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.