How can we set a limit for max tokens in ConversationSummaryMemory

[m-ali-awan]

Hi all, One major benefit of using Summary method for context in Conversation is to save cost. But with increasing chat iterations, no of token keeps on increasing, significantly. Is there any parameter, by which I can set the max limit of Summary?

[giuice]

I think the solution is use ConversationSummaryBufferMemory

from langchain.chains import ConversationChain
conversation_with_summary = ConversationChain(
    llm=llm, 
    # We set a very low max_token_limit for the purposes of testing.
    memory=ConversationSummaryBufferMemory(llm=OpenAI(), max_token_limit=40),
    verbose=True
)
conversation_with_summary.predict(input="Hi, what's up?")

[m-ali-awan]

Thanks @giuice

[dosubot[bot]]

Hi, @m-ali-awan! I'm Dosu, and I'm here to help the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue is about setting a limit for the maximum number of tokens in ConversationSummaryMemory. You asked if there is a parameter available to set this maximum limit. Another user named "giuice" suggested using ConversationSummaryBufferMemory as a solution and even provided an example code snippet. You thanked "giuice" for the suggestion.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself or it will be automatically closed in 7 days.

Thank you for your understanding and contribution to the LangChain project!

[celikmustafa89]

How can I put limit in prediction instead of initialisation