llama.cpp icon indicating copy to clipboard operation
llama.cpp copied to clipboard

Truncate prompt if longer than context + n_predict

Open matthew-mcallister opened this issue 1 year ago • 0 comments

I think this is an improvement over the current behavior of outputting nothing at all when the prompt is too long. It's slightly ugly to see the truncated prompt in the terminal, but it is helpful to see exactly what was included in the prompt so you can tell what was truncated.

Maybe there's a better solution, IDK, just throwing a PR your way.

matthew-mcallister avatar Mar 13 '23 04:03 matthew-mcallister