airllm
airllm copied to clipboard
ValueError: max() arg is an empty sequence(Apple M2 Max, macOS 14.2.1)
main.py
from airllm import AirLLMLlamaMlx
import mlx.core as mx
MAX_LENGTH = 128
# could use hugging face model repo id:
model = AirLLMLlamaMlx("Qwen/Qwen-7B-Chat",layer_shards_saving_path='.cache')
input_text = [
'I like',
]
input_tokens = model.tokenizer(input_text,
return_tensors="np",
return_attention_mask=False,
truncation=True,
max_length=MAX_LENGTH)
generation_output = model.generate(
mx.array(input_tokens['input_ids']),
max_new_tokens=3,
use_cache=True,
return_dict_in_generate=True)
print(generation_output)
python3 main.py
found index file...
found_layers:{'model.embed_tokens.': False, 'model.norm.': False, 'lm_head.': False}
some layer splits found, some are not, re-save all layers in case there's some corruptions.
0%| | 0/3 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/Users/tangjian/www/python/main.py", line 7, in <module>
model = AirLLMLlamaMlx("Qwen/Qwen-7B-Chat",layer_shards_saving_path='.cache')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/airllm/airllm_llama_mlx.py", line 224, in __init__
self.model_local_path, self.checkpoint_path = find_or_create_local_splitted_path(model_local_path_or_repo_id,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/airllm/utils.py", line 351, in find_or_create_local_splitted_path
return Path(model_local_path_or_repo_id), split_and_save_layers(model_local_path_or_repo_id, layer_shards_saving_path,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/airllm/utils.py", line 270, in split_and_save_layers
if max(shards) > shard:
^^^^^^^^^^^
ValueError: max() arg is an empty sequence
我也遇到了这个问题。
我也遇到了这个问题。
你用过官方推荐的那个70B的大模型没?我1T快用完了,没下那个大模型,直接用之前下载的7B了,用阿里官方示例是可以跑起来的,用这个不行
mac version doesn't support QWen yet. Only support Llama/Llama2 series models.
we'll add the support later.
mac version doesn't support QWen yet. Only support Llama/Llama2 series models.
we'll add the support later.
Thank you for your reply. I am looking forward to the day of support.
I got the same problem when using Qwen-70B-chat