ggml icon indicating copy to clipboard operation
ggml copied to clipboard

model unloads from memory after latest update

Open aseok opened this issue 1 year ago • 0 comments

Hi. Running starchat-beta and updated to latest ggml, the model unloads from memory after each prompt completion. command run from ggml/bin: ./bin/starcoder -m ../models/HuggingFaceH4/starchat-beta-ggml-q5_1.bin -p "prompt".

aseok avatar Jun 11 '23 14:06 aseok