Bug: Failed to get response: {"error":{"message":"prompt token count of 22298 exceeds the limit of 20000","param":"","code":"","type":""}}

[tecfu]

Issue:

After a certain threshold of input/output in the Copilot chat window, I get the following kind of error:

Failed to get response: {"error":{"message":"prompt token count of 22298 exceeds the limit of 20000","param":"","code":"","type":""}}

Fix Request:

Can we expose a command to truncate the tokens created subsequent to the oldest n entries until we can back under the prompt token threshold?

[tecfu]

Also wonder if it would be possible to call an external service to refactor/compress the tokens we've accumulated.

[pidgeon777]

I have found that this bug occasionally occurs in my environment as well. Hopefully I'll be able to catch the exact moment it happens, in order to provide a more detailed and accurate report.

[pidgeon777]

Also wonder if it would be possible to call an external service to refactor/compress the tokens we've accumulated.

Interesting idea.

[deathbeam]

I added auto trimming of history to #455, the oldest history message is simply discarded until theres enough space when asking new question.