llama.cpp
llama.cpp copied to clipboard
Truncate prompt if longer than context + n_predict
I think this is an improvement over the current behavior of outputting nothing at all when the prompt is too long. It's slightly ugly to see the truncated prompt in the terminal, but it is helpful to see exactly what was included in the prompt so you can tell what was truncated.
Maybe there's a better solution, IDK, just throwing a PR your way.