spring-ai
spring-ai copied to clipboard
Introduce first-class chat memory support
trafficstars
- ChatMemory will become a generic interface to implement different memory management strategies. It’s been moved from the “”spring-ai-client-chat” package to “spring-ai-model” package while retaining the same package, so it’s transparent to users.
- A MessageWindowChatMemory has been introduced to provide support for a chat memory that keeps at most N messages in the memory.
- A MessageWindowProcessingPolicy API has been introduced to customise the processing policy for the message window. A default implementation is provided out-of-the-box.
- A ChatMemoryRepository interface has been introduced to support different storage strategies for the chat memory. It’s meant to be used as part of a ChatMemory implementation. This is different than before, where the storage-specific implementation was directly tied to the ChatMemory. This design is familiar to Spring users since it’s used already in the ecosystem. The goal was to use a programming model similar to Spring Session and Spring Data.
- The JdbcChatMemory has been supersed by JdbcChatMemoryRepository.
- The ChatClient now supports memory as a first-class citizen, superseding the need for an Advisor to manage the chat memory. It also simplifies providing a conversationId. This feature lays the foundation for including the intermediate messages in tool calling in the memory as well.
- All the changes introduced in this PR are backword-compatible.
Usage:
ChatClient chatClient = ChatClient.builder(this.chatModel)
.defaultMemory(MessageWindowChatMemory.builder().build())
.build();
String conversationId = "007";
ChatResponse response1 = chatClient.prompt("My name is Bond. James Bond.")
.conversationId(conversationId)
.call()
.chatResponse(); // Hello, James Bond!
ChatResponse response2 = chatClient.prompt("What is my name?")
.conversationId(conversationId)
.call()
.chatResponse(); // Your name is James Bond!