mllm
mllm copied to clipboard
The Llama 7B model works on my Android phone, but other models do not.
Hello, I've been asking a lot of questions today. After building the Android phone app I created as an example and installing it on a Galaxy S22 model with 12GB of memory, I found that only the Llama 7B model works properly, while other models either cause the app to crash or only respond with repeated <unk><unk><unk><unk><unk><unk><unk><unk>....
The models I tested were Gemma-2b-Q4_K and Mistral-7b-Q4_K, both of which worked well in the PC demo. Is there any variable or setting I need to adjust to apply models other than Llama on Android? Below is the code I modified while switching models.
update : I have noted in the documentation that the current Android app is in a very early stage and only supports the Llama2 and Fuyu models. Do you have any plans to support other models like Gemma? Also, I am curious if it would be possible for Llama2 to work with models that have been fine-tuned for specific languages, such as Llama2-ko. I had many questions, but I sincerely thank you in advance for your prompt response.
[LibHelper.hpp]
enum PreDefinedModel {
LLAMA = 0,
FUYU,
};
[MainActivity.kt]
fun initStatus(context: Context,modelType:Int=_modelType.value?:0){
if (_isExternalStorageManager.value != true) return;
val modelPath = when(modelType){
0->"model/llama-2-7b-chat-q4_k.mllm"
// 0->"model/gemma-2b-q4_k.mllm"
// 0->"model/model_q4_k.mllm"
// 0->"model/model.mllm"
// 0->"model/mistral-7b-q4_k.mllm"
1->"model/llama_2.mllm"
2->"model/fuyu.mllm"
else->"model/llama"
}
val vacabPath = when(modelType){
0->"model/llama_vocab.mllm"
// 0->"model/gemma_vocab.mllm"
// 0->"model/vocab.mllm"
// 0->"model/mistral_vocab.mllm"
1->"model/vocab_uni.mllm"
else->"model/vocab.mllm"
}
var downloadsPath = "/storage/emulated/0/Download/"
if (!downloadsPath.endsWith("/")){
downloadsPath = downloadsPath.plus("/")