tabby icon indicating copy to clipboard operation
tabby copied to clipboard

llama-server exited with status code -1

Open kba-tmn3 opened this issue 1 year ago • 0 comments

Describe the bug I would like to start DeepseekCoderV2 on my PC, windows + WSL, it's worked with other models already. Now I have a message about llama server

 WARN llama_cpp_server::supervisor: crates/llama-cpp-server/src/supervisor.rs:99: llama-server <embedding> exited with status code -1

Restart and reload models cleaning models directory does not help. I need to fix it and I don't know how

Information about your version tabby 0.13.1 (I tried 0.14.0, nothing changes)

Information about your GPU

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 546.65                 Driver Version: 546.65       CUDA Version: 12.3     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                     TCC/WDDM  | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 4060 Ti   WDDM  | 00000000:01:00.0  On |                  N/A |
|  0%   48C    P2              35W / 165W |   2065MiB / 16380MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A       636    C+G   ...3.0_x64__t4vj0pshhgkwm\Telegram.exe    N/A      |
|    0   N/A  N/A      5552    C+G   C:\Windows\explorer.exe                   N/A      |
|    0   N/A  N/A      7232    C+G   ....Search_cw5n1h2txyewy\SearchApp.exe    N/A      |
|    0   N/A  N/A      8968    C+G   ...CBS_cw5n1h2txyewy\TextInputHost.exe    N/A      |
|    0   N/A  N/A     10332    C+G   ...5n1h2txyewy\ShellExperienceHost.exe    N/A      |
|    0   N/A  N/A     18120    C+G   ...\Docker\frontend\Docker Desktop.exe    N/A      |
|    0   N/A  N/A     18964    C+G   ...t.LockApp_cw5n1h2txyewy\LockApp.exe    N/A      |
|    0   N/A  N/A     26184    C+G   ...ndexBrowser\Application\browser.exe    N/A      |
|    0   N/A  N/A     28648    C+G   ...al\Discord\app-1.0.9157\Discord.exe    N/A      |
+---------------------------------------------------------------------------------------+

Additional context I restarted Docker daemon fully and it helped once but after this temporary fix it crushes again and don't work at all! with 0.13.1

docker run -it --gpus all -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby:0.13.1 serve --model kba-tmn3/DeepseekCoder-V2-L --device cuda

with 0.14..0

docker run -it --gpus all -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby:0.14.0 serve --model kba-tmn3/DeepseekCoder-V2-L --device cuda

Now I can't back to previous models, it crushes constantly!

kba-tmn3 avatar Aug 08 '24 05:08 kba-tmn3