ggml
ggml copied to clipboard
model unloads from memory after latest update
Hi. Running starchat-beta and updated to latest ggml, the model unloads from memory after each prompt completion. command run from ggml/bin: ./bin/starcoder -m ../models/HuggingFaceH4/starchat-beta-ggml-q5_1.bin -p "prompt".