chatglm.cpp icon indicating copy to clipboard operation
chatglm.cpp copied to clipboard

Allow configuring SCRATCH_SIZE (not enough space in the scratch memory pool)

Open kardianos opened this issue 11 months ago • 0 comments

After chatting for a few lines, I receive the following error:

ChatGLM2 > ggml_new_tensor_impl: not enough space in the scratch memory pool (needed 173910528, available 150994944) /home/d/bin/chat-gml2: line 4: 4131493 Segmentation fault (core dumped) $GLM/build/bin/main -m $MODEL -i -t 8 -c 2048 -l 8000

This can be resolved by increasing the SCRATCH_SIZE in chatglm.h.

I'm creating this issue for 2 reasons: 1. To help someone else resolve this if they encounter it, and 2. request a knob for the value.

kardianos avatar Jul 05 '23 15:07 kardianos