jupyter-ai icon indicating copy to clipboard operation
jupyter-ai copied to clipboard

Chat memory is reset when selecting new LLM

Open michaelchia opened this issue 1 year ago • 0 comments
trafficstars

Description

Chat memory is reset when selecting new LLM provider.

https://github.com/jupyterlab/jupyter-ai/blob/3bfce328e3b6f730d05faa68ca9b6d6434b5fdec/packages/jupyter-ai/jupyter_ai/chat_handlers/default.py#L34-L36

This is due to self.memory being reassigned in DefaultChatHandler.create_llm_chain which is triggered when LLM provider is changed.

Expected behavior

I want to be able to change LLM provider while continuing a conversation. Separate reset command can be provided to allow user to reset history if desired (https://github.com/jupyterlab/jupyter-ai/issues/616).

Suggested solution

Fix would be to remove L34-L36 above which would reuse the existing self.memory object instantiated in L21 in DefaultChatHandler.__init__. Also this current approach makes that instantiation in DefaultChatHandler.__init__ useless as it will always be overriden in DefaultChatHandler.create_llm_chain anyway.

michaelchia avatar Apr 05 '24 09:04 michaelchia