tabby
tabby copied to clipboard
llama-server exited with status code -1
Describe the bug I would like to start DeepseekCoderV2 on my PC, windows + WSL, it's worked with other models already. Now I have a message about llama server
WARN llama_cpp_server::supervisor: crates/llama-cpp-server/src/supervisor.rs:99: llama-server <embedding> exited with status code -1
Restart and reload models cleaning models directory does not help. I need to fix it and I don't know how
Information about your version
tabby 0.13.1 (I tried 0.14.0, nothing changes)
Information about your GPU
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 546.65 Driver Version: 546.65 CUDA Version: 12.3 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 4060 Ti WDDM | 00000000:01:00.0 On | N/A |
| 0% 48C P2 35W / 165W | 2065MiB / 16380MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 636 C+G ...3.0_x64__t4vj0pshhgkwm\Telegram.exe N/A |
| 0 N/A N/A 5552 C+G C:\Windows\explorer.exe N/A |
| 0 N/A N/A 7232 C+G ....Search_cw5n1h2txyewy\SearchApp.exe N/A |
| 0 N/A N/A 8968 C+G ...CBS_cw5n1h2txyewy\TextInputHost.exe N/A |
| 0 N/A N/A 10332 C+G ...5n1h2txyewy\ShellExperienceHost.exe N/A |
| 0 N/A N/A 18120 C+G ...\Docker\frontend\Docker Desktop.exe N/A |
| 0 N/A N/A 18964 C+G ...t.LockApp_cw5n1h2txyewy\LockApp.exe N/A |
| 0 N/A N/A 26184 C+G ...ndexBrowser\Application\browser.exe N/A |
| 0 N/A N/A 28648 C+G ...al\Discord\app-1.0.9157\Discord.exe N/A |
+---------------------------------------------------------------------------------------+
Additional context I restarted Docker daemon fully and it helped once but after this temporary fix it crushes again and don't work at all! with 0.13.1
docker run -it --gpus all -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby:0.13.1 serve --model kba-tmn3/DeepseekCoder-V2-L --device cuda
with 0.14..0
docker run -it --gpus all -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby:0.14.0 serve --model kba-tmn3/DeepseekCoder-V2-L --device cuda
Now I can't back to previous models, it crushes constantly!