continue icon indicating copy to clipboard operation
continue copied to clipboard

Show warning message when last user input get pruned

Open Jazzcort opened this issue 7 months ago • 27 comments

Description

If the user's last input is pruned due to context overflow, a warning message will be displayed in the chat section, alerting them that some details may have been lost. As a result, the response they receive might be incomplete or inaccurate due to the truncated input. https://github.com/Granite-Code/granite-code/issues/22

Screenshots

show-warning-in-chat

Testing instructions

Set the model’s context length to a small value (e.g., 512) and ask a question that exceeds contextLength - maxTokens tokenwise. A warning message will appear at the bottom of the chat section, indicating that some input may have been truncated. Deleting previous messages will remove the warning.

Jazzcort avatar Mar 25 '25 18:03 Jazzcort