Feat/history summarizer
Personally, I think this code I wrote here as-is great for proof of concept and testing, but for production, we would have to decide if we are going to completely do away with how the message history/context token usage is optimized (deleting older messages after a certain token usage threshold) or somehow fit this technique in with the truncate/lobotomize method. It might also be a good idea to dynamically set the max_tokens value sent to the API. The minimum should be no lower than 400 tokens, as anything less makes the whole thing useless again in terms of context continuity.
So how are we doing with this code change, how would i test it if i try it out ?
We have Agent Summary that is doing the Summarization of the history. Please refer to the v0.0.10 release notes. Closing this PR.