mlc-llm
mlc-llm copied to clipboard
mlc-ai
[Question] Running an Android APK on the Qualcomm 8155 chip, loading phi-2-q4f16_1 results in a CL_OUT_OF_RESOURCES error, while it loads successfully on a mobile phone.
❓ General Questions
Running an Android APK on the Qualcomm 8155 chip, loading phi-2-q4f16_1 results in a CL_OUT_OF_RESOURCES error, while it loads successfully on a mobile phone.
llm: phi-2-q4f16_1 device: Qualcomm 8155 Adb Log: android-log-phi-2-2024-02-22.log
Hey, which device are you using? And also are you using the prebuilt libs or compiling yourself? If compiling yourself, can you share your mlc-chat-config.json file as well?
The full name of Qualcomm 8155 is Qualcomm Snapdragon SA8155P, which is an automotive-grade chip.
I can compile libs myself.
mlc-chat-config.json { "model_type": "phi-msft", "quantization": "q4f16_1", "model_config": { "vocab_size": 51200, "n_positions": 2048, "n_embd": 2560, "n_layer": 32, "n_inner": 10240, "n_head": 32, "rotary_dim": 32, "position_embedding_base": 10000, "layer_norm_epsilon": 1e-05, "context_window_size": 2048, "prefill_chunk_size": 2048, "n_head_kv": 32, "head_dim": 80, "tensor_parallel_shards": 1 }, "vocab_size": 51200, "context_window_size": 2048, "sliding_window_size": -1, "prefill_chunk_size": 2048, "attention_sink_size": -1, "tensor_parallel_shards": 1, "mean_gen_len": 128, "max_gen_len": 512, "shift_fill_factor": 0.3, "temperature": 0.7, "repetition_penalty": 1.0, "top_p": 0.95, "conv_template": "phi-2", "pad_token_id": 0, "bos_token_id": 1, "eos_token_id": 2, "tokenizer_files": [ "tokenizer.json", "vocab.json", "merges.txt", "added_tokens.json", "tokenizer_config.json" ], "version": "0.1.0" }
❓ General Questions
Running an Android APK on the Qualcomm 8155 chip, loading phi-2-q4f16_1 results in a CL_OUT_OF_RESOURCES error, while it loads successfully on a mobile phone.
llm: phi-2-q4f16_1 device: Qualcomm 8155 Adb Log: android-log-phi-2-2024-02-22.log
Hello, may I ask if you have found the reason. I encountered a similar problem
