adel boussaken
adel boussaken
@LukeMoody01 I could run tinyllama-2-1b-miniguanaco.Q3_K_L.gguf with script in example folder, both simple.dart and chat.dart --- could it be that your prompt is larger than context? I will try investigate also...
@LukeMoody01 please try again
thanks, in llama class, I have predict field fixed at low value, this set length of output, I will expose it On Wed, Nov 27, 2024, 01:34 Luke Moody ***@***.***>...
tested those models https://huggingface.co/TheBloke/Tinyllama-2-1b-miniguanaco-GGUF/blob/main/tinyllama-2-1b-miniguanaco.Q3_K_L.gguf https://huggingface.co/mradermacher/Qwen2-7B-Multilingual-RP-GGUF/blob/main/Qwen2-7B-Multilingual-RP.Q8_0.gguf https://huggingface.co/MaziyarPanahi/gemma-7b-GGUF/blob/main/gemma-7b.Q8_0.gguf
@LukeMoody01 I am currently testing on ios, how do you build llama.cpp for ios?
sorry I just noticed PRs, I will look into them
@hacker1024 sure, I will do in next release
thank you @danemadsen I will investigate this
I have tested `tinyllama-2-1b-miniguanaco.Q3_K_L.gguf` with last `llama.cpp` I got text correctly generated ```bash dart example/chat.dart llama_model_loader: loaded meta data with 20 key-value pairs and 201 tensors from /Users/adel/Workspace/llama.cpp/models/tinyllama-2-1b-miniguanaco.Q3_K_L.gguf (version GGUF...
directly, not LlamaProcessor... you can see code here https://github.com/netdur/llama_cpp_dart/blob/main/example/chat.dart during merges I have also fixed some issues and updated bindings a bit, I accidentally fixed it