serge-chat
LLAMA binary will sometimes just crash and produce no logs.
Bug description
I have been playing around with this for a few hours now and it has been a lot of fun! Getting to use 20 out of my 24 cores (R9 3900X) and a good bunch of my 32GB of RAM has been entertaining, for sure!
However, even after increasing WSL's resource limits, LLAMA will still sometimes just crash.
Steps to reproduce
- Enable WSL2, Install Docker, enable wsl2 backend.
- Follow readme to build and start the container.
- Download 7b and 7b-native.
- Have a few interactions with the model.
- Observe forever spinning circle.
-
wsl -d docker-desktop apk add htop -
wsl -d docker-desktop htopin another window. - Try to send a "Continue" message.
- In
htop: Observellamastarting and seemingly spinning up as all 20 threads increase usage. - Immediately after, the process and usage disappear.
- Check
docker-compose logsand find no output.
(I wouldn't be surprised if it printed at least an error to stderr...)
Environment Information
Docker: Docker version 20.10.23, build 7155243 OS: Windows 10 22h2 (Build 19045.2788) CPU: AMD Ryzen 9 3900X RAM: 32GB @ 2133Mhz
.wslconfig:
[wsl2]
memory=20G
processors=20
Screenshots
Infinite spinner:
Sadly I don't know how to make a gif screenshot of just how fast the process dies... Sorry.
Relevant log output
None, which is the odd part. o.o
Confirmations
- [X] I'm running the latest version of the main branch.
- [X] I checked existing issues to see if this has already been described.
Retested with the updated llama.cpp master update in the Dockerfile - it still takes an angry nap :)
Suggestion: Option (env variable?) to print stdout/stderr upon task completion. Should enable better debugging.
