ESMC AttributeError

[Mgctz]

from esm.models.esmc import ESMC client = ESMC.from_pretrained("esmc_300m").to("cpu") Traceback (most recent call last): File "", line 1, in File "/home/work/mindformers/esm_env/lib/python3.10/site-packages/esm/models/esmc.py", line 86, in from_pretrained model = load_local_model(model_name, device=device) File "/home/work/mindformers/esm_env/lib/python3.10/site-packages/esm/pretrained.py", line 135, in load_local_model return LOCAL_MODEL_REGISTRYmodel_name File "/home/work/mindformers/esm_env/lib/python3.10/site-packages/esm/pretrained.py", line 71, in ESMC_300M_202412 tokenizer=get_esmc_model_tokenizers(), File "/home/work/mindformers/esm_env/lib/python3.10/site-packages/esm/tokenization/init.py", line 52, in get_esmc_model_tokenizers return EsmSequenceTokenizer() File "/home/work/mindformers/esm_env/lib/python3.10/site-packages/esm/tokenization/sequence_tokenizer.py", line 55, in init super().init( File "/home/work/mindformers/esm_env/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 177, in init super().init(**kwargs) File "/home/work/mindformers/esm_env/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1634, in init super().init(**kwargs) File "/home/work/mindformers/esm_env/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 890, in init setattr(self, key, value) AttributeError: can't set attribute 'cls_token'

[auroua]

Just comment the following lines of codes in the esm/tokenization/sequence_tokenizer.py file will fix the issue. As this class only provids get method for these attributes which blocks the super class set these attributes during the intialization of the super class.

    # @property
    # def cls_token(self):
    #     return self._get_token("cls_token")

    # @property
    # def eos_token(self):
    #     return self._get_token("eos_token")

    # @property
    # def mask_token(self):
    #     return self._get_token("mask_token")

    # @property
    # def pad_token(self):
    #     return self._get_token("pad_token")

Futher more, the following line of codes also need to be added.

    @chain_break_token.setter
    def chain_break_token(self, cb_token):
        self.cb_token = cb_token

After executing the above modification, the code can run correctly.

[bviggiano]

Also ran into this! https://github.com/evolutionaryscale/esm/issues/214