mlx-examples
mlx-examples copied to clipboard
Issue with fetch_from_hub tokenizer for fine-tuned models
When trying to convert fine-tuned weights of StarCode2 via
python -m mlx_lm.convert \
--hf-path m-a-p/OpenCodeInterpreter-SC2-7B \
-q \
--upload-repo mlx-community/OpenCodeInterpreter-SC2-7B-4bit
I encountered
-> tokenizer = AutoTokenizer.from_pretrained(model_path_hf)
return cls._from_pretrained(
File "/Users/x/miniconda3/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2082, in _from_pretrained
slow_tokenizer = (cls.slow_tokenizer_class)._from_pretrained(
File "/Users/x/miniconda3/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2287, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
File "/Users/x/miniconda3/lib/python3.10/site-packages/transformers/models/gpt2/tokenization_gpt2.py", line 187, in __init__
with open(merges_file, encoding="utf-8") as merges_handle:
**TypeError: expected str, bytes or os.PathLike object, not NoneType**
I printed path and config to inspect
/Users/x/.cache/huggingface/hub/models--m-a-p--OpenCodeInterpreter-SC2-7B/snapshots/ddf1d84dfc2adbc7ee9eed97298de13fc232c300
Config: Starcoder2Config {
"_name_or_path":
"/Users/x/.cache/huggingface/hub/models--m-a-p--OpenCodeInterpreter-SC2-7B/snapshots/ddf1d84dfc2adbc7ee9eed97298de13fc232c30
0",
After changing model_path
to non snapshotted original name issue disappeared
from rich import print as rprint
def fetch_from_hub(
model_path: Path, lazy: bool = False
) -> Tuple[nn.Module, dict, PreTrainedTokenizer]:
model = load_model(model_path, lazy)
print(model_path)
config = AutoConfig.from_pretrained(model_path)
rprint(f"Config: {config}")
_model_path_hf = "m-a-p/OpenCodeInterpreter-SC2-7B"_
tokenizer = AutoTokenizer.from_pretrained(model_path_hf)
return model, config.to_dict(), tokenizer
@awni maybe utils.py
should not modify "_name_or_path":
I actually just tried to use convert on the model and I got this issue python -m mlx_lm.convert --hf-path bigcode/starcoder2-3b
:
File "/Users/awni/mlx-examples/llms/mlx_lm/utils.py", line 413, in fetch_from_hub
config = AutoConfig.from_pretrained(model_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/awni/miniconda3/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1130, in from_pretrained
raise ValueError(
ValueError: The checkpoint you are trying to load has model type `starcoder2` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
Which Transformer version are you using?
I actually just tried to use convert on the model and I got this issue
python -m mlx_lm.convert --hf-path bigcode/starcoder2-3b
:File "/Users/awni/mlx-examples/llms/mlx_lm/utils.py", line 413, in fetch_from_hub config = AutoConfig.from_pretrained(model_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/awni/miniconda3/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1130, in from_pretrained raise ValueError( ValueError: The checkpoint you are trying to load has model type `starcoder2` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
Which Transformer version are you using?
Startcoder2 needs the transformer built from the master, which includes the model and configuration support. not in release build yet.
I actually just tried to use convert on the model and I got this issue
python -m mlx_lm.convert --hf-path bigcode/starcoder2-3b
:File "/Users/awni/mlx-examples/llms/mlx_lm/utils.py", line 413, in fetch_from_hub
config = AutoConfig.from_pretrained(model_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/awni/miniconda3/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1130, in from_pretrained
raise ValueError(
ValueError: The checkpoint you are trying to load has model type
starcoder2
but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.Which Transformer version are you using?
Startcoder2 needs the transformer built from the master, which includes the model and configuration support. not in release build yet.
I am using the version built from source
Fix is here https://github.com/ml-explore/mlx-examples/pull/574