mlx-examples
mlx-examples copied to clipboard
Can't generate token with lora adapter
Hi Team, Just solved the training problem last night, but this time I failed to evaluate the the trained lora adapter with the following command
python3 -m mlx_lm.lora --model ~/Models/mixtral-8x7B-v0.1 \
--adapter-file ~/Models/adapters/mixstral-8x7B-ft.npz \
--prompt "table: 1-10015132-16 columns: Player, No., Nationality, Position, Years in Toronto, Schoo/Club Team \
Q: What is terrence ross' nationality \
A:"
I got a strange error
lora.py: error: unrecognized arguments: --prompt table: 1-10015132-16 columns: Player, No., Nationality, Position, Years in Toronto, Schoo/Club Team Q: What is terrence ross' nationality
also the fuse can't work either
python3 -m mlx_lm.fuse --model ~/Models/mixtral-8x7B-v0.1 --adapter-file ~/Models/adapters/mixstral-8x7B-ft.npz --save-path ~/Models/mixtral-ft
Traceback (most recent call last):
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/Users/linz/Library/Python/3.9/lib/python/site-packages/mlx_lm/fuse.py", line 9, in <module>
from .gguf import convert_to_gguf
File "/Users/linz/Library/Python/3.9/lib/python/site-packages/mlx_lm/gguf.py", line 24, in <module>
class HfVocab:
File "/Users/linz/Library/Python/3.9/lib/python/site-packages/mlx_lm/gguf.py", line 26, in HfVocab
self, fname_tokenizer: Path, fname_added_tokens: Path | None = None
TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'