Issue with fetch_from_hub tokenizer for fine-tuned models

[Muhtasham]

When trying to convert fine-tuned weights of StarCode2 via

python -m mlx_lm.convert \
    --hf-path m-a-p/OpenCodeInterpreter-SC2-7B \
    -q \
    --upload-repo mlx-community/OpenCodeInterpreter-SC2-7B-4bit

I encountered

->    tokenizer = AutoTokenizer.from_pretrained(model_path_hf)
    return cls._from_pretrained(
  File "/Users/x/miniconda3/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2082, in _from_pretrained
    slow_tokenizer = (cls.slow_tokenizer_class)._from_pretrained(
  File "/Users/x/miniconda3/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2287, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/Users/x/miniconda3/lib/python3.10/site-packages/transformers/models/gpt2/tokenization_gpt2.py", line 187, in __init__
    with open(merges_file, encoding="utf-8") as merges_handle:
**TypeError: expected str, bytes or os.PathLike object, not NoneType**

I printed path and config to inspect

/Users/x/.cache/huggingface/hub/models--m-a-p--OpenCodeInterpreter-SC2-7B/snapshots/ddf1d84dfc2adbc7ee9eed97298de13fc232c300
Config: Starcoder2Config {
  "_name_or_path": 
"/Users/x/.cache/huggingface/hub/models--m-a-p--OpenCodeInterpreter-SC2-7B/snapshots/ddf1d84dfc2adbc7ee9eed97298de13fc232c30
0",

After changing model_path to non snapshotted original name issue disappeared

from rich import print as rprint

def fetch_from_hub(
    model_path: Path, lazy: bool = False
) -> Tuple[nn.Module, dict, PreTrainedTokenizer]:
    model = load_model(model_path, lazy)
    print(model_path)
    config = AutoConfig.from_pretrained(model_path)
    rprint(f"Config: {config}")
    _model_path_hf = "m-a-p/OpenCodeInterpreter-SC2-7B"_
    tokenizer = AutoTokenizer.from_pretrained(model_path_hf)

    return model, config.to_dict(), tokenizer

@awni maybe utils.py should not modify "_name_or_path":

[awni]

I actually just tried to use convert on the model and I got this issue python -m mlx_lm.convert --hf-path bigcode/starcoder2-3b:

  File "/Users/awni/mlx-examples/llms/mlx_lm/utils.py", line 413, in fetch_from_hub
    config = AutoConfig.from_pretrained(model_path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/awni/miniconda3/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1130, in from_pretrained
    raise ValueError(
ValueError: The checkpoint you are trying to load has model type `starcoder2` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

Which Transformer version are you using?

[mzbac]

I actually just tried to use convert on the model and I got this issue python -m mlx_lm.convert --hf-path bigcode/starcoder2-3b:

  File "/Users/awni/mlx-examples/llms/mlx_lm/utils.py", line 413, in fetch_from_hub
    config = AutoConfig.from_pretrained(model_path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/awni/miniconda3/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1130, in from_pretrained
    raise ValueError(
ValueError: The checkpoint you are trying to load has model type `starcoder2` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

Which Transformer version are you using?

Startcoder2 needs the transformer built from the master, which includes the model and configuration support. not in release build yet.

[Muhtasham]

I actually just tried to use convert on the model and I got this issue python -m mlx_lm.convert --hf-path bigcode/starcoder2-3b:

File "/Users/awni/mlx-examples/llms/mlx_lm/utils.py", line 413, in fetch_from_hub

config = AutoConfig.from_pretrained(model_path)

         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/Users/awni/miniconda3/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1130, in from_pretrained

raise ValueError(

ValueError: The checkpoint you are trying to load has model type starcoder2 but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

Which Transformer version are you using?

Startcoder2 needs the transformer built from the master, which includes the model and configuration support. not in release build yet.

I am using the version built from source

[awni]

Fix is here https://github.com/ml-explore/mlx-examples/pull/574