Anima ValueError: max() arg is an empty sequence(Apple M2 Max, macOS 14.2.1)

ValueError: max() arg is an empty sequence(Apple M2 Max, macOS 14.2.1)

Open tvsj opened this issue 6 months ago • 6 comments

main.py

from airllm import AirLLMLlamaMlx
import mlx.core as mx

MAX_LENGTH = 128
# could use hugging face model repo id:
model = AirLLMLlamaMlx("Qwen/Qwen-7B-Chat",layer_shards_saving_path='.cache')

input_text = [
        'I like',
    ]

input_tokens = model.tokenizer(input_text,
    return_tensors="np", 
    return_attention_mask=False, 
    truncation=True, 
    max_length=MAX_LENGTH)
           
generation_output = model.generate(
    mx.array(input_tokens['input_ids']), 
    max_new_tokens=3,
    use_cache=True,
    return_dict_in_generate=True)

print(generation_output)

python3 main.py

found index file...
found_layers:{'model.embed_tokens.': False, 'model.norm.': False, 'lm_head.': False}
some layer splits found, some are not, re-save all layers in case there's some corruptions.
  0%|                                                                                                                                                     | 0/3 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/Users/tangjian/www/python/main.py", line 7, in <module>
    model = AirLLMLlamaMlx("Qwen/Qwen-7B-Chat",layer_shards_saving_path='.cache')
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/airllm/airllm_llama_mlx.py", line 224, in __init__
    self.model_local_path, self.checkpoint_path = find_or_create_local_splitted_path(model_local_path_or_repo_id,
                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/airllm/utils.py", line 351, in find_or_create_local_splitted_path
    return Path(model_local_path_or_repo_id), split_and_save_layers(model_local_path_or_repo_id, layer_shards_saving_path,
                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/airllm/utils.py", line 270, in split_and_save_layers
    if max(shards) > shard:
       ^^^^^^^^^^^
ValueError: max() arg is an empty sequence