feature: Automatically Generates Conversation Summaries to Prevent Exceeding Token Usage Limits

[llegomark]

1. Sending chat history is a bit costly. We could use a separate ChatGPT instance to create a new completion request to summarize and shorten it after its size reaches a cutoff value. This context can come from the last few messages in the conversation. The GPT model can then use this context to generate a more accurate and relevant response.

Reference: https://platform.openai.com/docs/guides/chat/introduction

2. We can also utilize OpenAI Embeddings, which can assist in storing conversation data as a means of enhancing long-term memory. Depending on the cost and purpose, this approach can be up to 5 to 9 times more economical compared to transmitting chat history.

Reference: https://github.com/LagPixelLOL/ChatGPTCLIBot

3. Drop old messages

Originally posted by me at https://discord.com/channels/1082263386080940053/1082339009302376449/1082515550757998672