chatglm.cpp
chatglm.cpp copied to clipboard
Allow configuring SCRATCH_SIZE (not enough space in the scratch memory pool)
After chatting for a few lines, I receive the following error:
ChatGLM2 > ggml_new_tensor_impl: not enough space in the scratch memory pool (needed 173910528, available 150994944) /home/d/bin/chat-gml2: line 4: 4131493 Segmentation fault (core dumped) $GLM/build/bin/main -m $MODEL -i -t 8 -c 2048 -l 8000
This can be resolved by increasing the SCRATCH_SIZE in chatglm.h.
I'm creating this issue for 2 reasons: 1. To help someone else resolve this if they encounter it, and 2. request a knob for the value.