maximum context length

[aymenfurter]

trafficstars

max context length needs to be taken into account, to avoid the following problem: 🤔 2023/03/26 09:08:10 error, status code: 400, message: This model's maximum context length is 4097 tokens. However, your messages resulted in 5616 tokens. Please reduce the length of the messages.

Perhaps deploying summarization techniques or previous steps?

[obra]

This is definitely biting me on the regular. It happens pretty reliably on git commits of initial content of an app and when it hits a stack trace.

I made some headway by augmenting the prompt to tell it to run git in quiet mode and to split steps into smaller parts. I wonder if it'd be possible to simply truncate input back to the LLM as a "simplest thing that could possibly work" half-measure.