mllm icon indicating copy to clipboard operation
mllm copied to clipboard

The Llama 7B model works on my Android phone, but other models do not.

Open siz0001 opened this issue 5 months ago • 2 comments

Hello, I've been asking a lot of questions today. After building the Android phone app I created as an example and installing it on a Galaxy S22 model with 12GB of memory, I found that only the Llama 7B model works properly, while other models either cause the app to crash or only respond with repeated <unk><unk><unk><unk><unk><unk><unk><unk>.... The models I tested were Gemma-2b-Q4_K and Mistral-7b-Q4_K, both of which worked well in the PC demo. Is there any variable or setting I need to adjust to apply models other than Llama on Android? Below is the code I modified while switching models.

update : I have noted in the documentation that the current Android app is in a very early stage and only supports the Llama2 and Fuyu models. Do you have any plans to support other models like Gemma? Also, I am curious if it would be possible for Llama2 to work with models that have been fine-tuned for specific languages, such as Llama2-ko. I had many questions, but I sincerely thank you in advance for your prompt response.

[LibHelper.hpp]

    enum PreDefinedModel {
        LLAMA = 0,
        FUYU,
    };

[MainActivity.kt]


    fun initStatus(context: Context,modelType:Int=_modelType.value?:0){
        if (_isExternalStorageManager.value != true) return;
        val modelPath = when(modelType){
         0->"model/llama-2-7b-chat-q4_k.mllm"

//           0->"model/gemma-2b-q4_k.mllm"
//            0->"model/model_q4_k.mllm"
//            0->"model/model.mllm"
//            0->"model/mistral-7b-q4_k.mllm"
            1->"model/llama_2.mllm"
            2->"model/fuyu.mllm"
            else->"model/llama"
        }
        val vacabPath = when(modelType){
          0->"model/llama_vocab.mllm"
//          0->"model/gemma_vocab.mllm"
//          0->"model/vocab.mllm"
//            0->"model/mistral_vocab.mllm"
            1->"model/vocab_uni.mllm"
            else->"model/vocab.mllm"
        }
        var downloadsPath = "/storage/emulated/0/Download/"
        if (!downloadsPath.endsWith("/")){
            downloadsPath = downloadsPath.plus("/")

siz0001 avatar Sep 14 '24 08:09 siz0001