dalai
dalai copied to clipboard
cocktailpeanut
I've done all of the above fixes. and now when I do the prompt I get:
I've done all of the above fixes. and now when I do the prompt I get:
[2J[m[HWindows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.
Try the new cross-platform PowerShell https://aka.ms/pscore6]0;C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe[?25h[25l[6;1H[?25hPS C:\Users\Adam\dalai\alpaca> [?25l[93mC[?25h[m[?25l PS C:\Users\Adam\dalai\alpaca> [93mC:\Users\Adam\dalai\alpaca\build\Release\main[m [90m--seed[m -1 [90m--threads[m [97m4[m [90m--n_predict[m [97m200[m [90m--model[m models/7B/ggml-model-q4_0.bin [90m--top_k[m [97m40[m [90m--top_p[m [97m0.9[m [90m--temp[m [97m0.1[m [90m--repeat_last
_n[m [97m64[m [90m--repeat_penalty[m [97m1.3[m [90m-p[m [36m"where is rome"[m[K
[?25hmain: seed = 1679321497
llama_model_load: loading model from 'models/7B/ggml-model-q4_0.bin' - please wait ...
llama_model_load: ggml ctx size = 6065.34 MB[14X
llama_model_load: memory_size = 2048.00 MB, n_mem = 65536
llama_model_load: loading model part 1/1 from 'models/7B/ggml-model-q4_0.bin'
llama_model_load: [59X.................................... done
[25lllama_model_load: model size = 4017.27 MB / num tensors = 291[113X
[175X
system_info: n_threads = 4 / 4 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 0 | VSX = 0 |
[175X
main: prompt: 'where is rome'[146X
main: number of tokens in prompt = 5[139X
1 -> ''[163X
3062 -> 'where'[158X
338 -> ' is'[160X
29871 -> ' '[162X
4871 -> 'rome'[159X
[175X
sampling parameters: temp = 0.100000, top_k = 40, top_p = 0.900000, repeat_last_n = 64, repeat_penalty = 1.300000[62X[29;1H[?25hPS C:\Users\Adam\dalai\alpaca> [93me[?25l[m[92mexit
[?25h
Originally posted by @nigelp in https://github.com/cocktailpeanut/dalai/issues/128#issuecomment-1476310763
I am cracking my head on the wall with this issue.... and I can't seem to get the llama files...
I have the exact same issue, and I'm not using Power Shell at all, just the regular command prompt.
I uninstalled everything and repeated the whole process twice. I always get that same weird result described above
Okay after running through the logs and the programs yargs was missing in the bin these commands sorted it out for me good luck
cd %USERPROFILE%\node_modules\dalai\bin npm install --save yargs
C:\Users&&&&\dalai-main\bin>npx dalai serve [#########.........] | idealTree: timing idealTree Completed in 326ms
Now I am getting this
I think I've found the issue, even though I'm launching this with cmd it's using PowerShell when it runs the autocomplete. Checking the debug option shows this.
Actually it might not be powershell that's the issue.
Running the command that debug spat out I get the same result in cmd myself.
C:\Users\xo\dalai\llama>C:\Users\xo\dalai\llama\build\Release\main --seed -1 --threads 4 --n_predict 20 --model models/7B/ggml-model-q4_0.bin --top_k 40 --top_p 0.9 --temp 0.1 --repeat_last_n 64 --repeat_penalty 1.3 -p " a small wizard lives in a forest"
main: seed = 1679363493
llama_model_load: loading model from 'models/7B/ggml-model-q4_0.bin' - please wait ...
llama_model_load: n_vocab = 32000
llama_model_load: n_ctx = 512
llama_model_load: n_embd = 4096
llama_model_load: n_mult = 256
llama_model_load: n_head = 32
llama_model_load: n_layer = 32
llama_model_load: n_rot = 128
llama_model_load: f16 = 2
llama_model_load: n_ff = 11008
llama_model_load: n_parts = 1
llama_model_load: ggml ctx size = 4529.34 MB
llama_model_load: memory_size = 512.00 MB, n_mem = 16384
llama_model_load: loading model part 1/1 from 'models/7B/ggml-model-q4_0.bin'
llama_model_load: done
llama_model_load: model size = 78.12 MB / num tensors = 1
system_info: n_threads = 4 / 12 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 0 | VSX = 0 |
main: prompt: ' a small wizard lives in a forest'
main: number of tokens in prompt = 10
1 -> ''
29871 -> ' '
263 -> ' a'
2319 -> ' small'
281 -> ' w'
17909 -> 'izard'
12080 -> ' lives'
297 -> ' in'
263 -> ' a'
13569 -> ' forest'
sampling parameters: temp = 0.100000, top_k = 40, top_p = 0.900000, repeat_last_n = 64, repeat_penalty = 1.300000
a small wizard lives in a forest
¶¶↓∟↑↓ ◄ ‼▬
main: mem per token = 14368644 bytes
main: load time = 30.53 ms
main: sample time = 17.84 ms
main: predict time = 5949.02 ms / 205.14 ms per token
main: total time = 6820.01 ms
Same results here, not using powershell.
Aha.. you got Llama 7B working.. I couldn't even get that in the Dalai... all I get is the Alpaca 7B... the other ones give me a timeout
I have the same issue. Only alpaca works for me. Weirdly enough the first time I tried it worked. Then I tried installing other models and it never worked again.
this worked for me: https://github.com/cocktailpeanut/dalai/issues/147#issuecomment-1476831396
this worked for me: #147 (comment)
Yes, I had found that post as well. I guess it messed up the files when I tried to download the 13b model and the download failed.
Thanks.
this worked for me: #147 (comment)
Yes. This was the thing. Thank you! I tried the same procedure even on different machines, and that finally solved it. I have no idea how could other windows users get these models running [could it be a different VS version? I'm using VS2022] By the way some people are mentioning that the alpaca model is working. That makes sense because in the build process, for alpaca, the final .exe is correctly called main.exe
this worked for me: #147 (comment)
Yes. This was the thing. Thank you! I tried the same procedure even on different machines, and that finally solved it. I have no idea how could other windows users get these models running [could it be a different VS version? I'm using VS2022] By the way some people are mentioning that the alpaca model is working. That makes sense because in the build process, for alpaca, the final .exe is correctly called main.exe
I don't think anyone got this working on Windows... I got Alpaca working on Zorin Linux on my laptop but the ram on that is only 8GB so llama doesn't load at all