Chat history in notebook magics for Chat model providers

[3coins]

trafficstars

Problem

The current magics implementation is using an artificial list of chat message history to keep track of exchanges between 2 or more cells when openai-chat model is used. Although, this might have the appearance of working as expected for OpenAI, this is not translatable and scalable to other chat model providers, for example ChatAnthropic.

Solution

Create a conversation chain and memory to generate messages for providers that support BaseChatModel. Using the memory class in an LLM chain automatically keeps track of message history and avoid manual tracking of exchanges. This also helps in future proofing the magics implementation for new chat providers that will be introduced in future.

[JasonWeill]

The OpenAI chat model history was removed in https://github.com/jupyterlab/jupyter-ai/pull/551, when we upgraded to a newer version of the OpenAI provider. If we reintroduce this feature, we should make it work with as many model providers as possible.