call-gpt
call-gpt copied to clipboard
Summarize GPT Chat history as more queries are added to the conversation
Currently, responses get increasingly slower as the chat context grows. We should implement a gradual rolloff of older queries that get summarized by GPT and then added to the end of the conversation.
This may require some testing to understand the optimal conversation context size based on quality of results and speed to response.