AWS Bedrock, persistent chat memory (jdbc and cassandra), and Spring AI's MessageChatMemoryAdvisor lead to 400 from Bedrock

[shariqh]

trafficstars

Bug description Whenever using JDBC or Cassandra for persistent chat memory, if the "ASSISTANT" is the latest message in a conversation (defined by chatId), the call Bedrock receives is malformed and gives us a 400. A subsequent message to the same chat after the error then works just fine - this cycle continues.

This issue does NOT appear with the in-memory tooling or with the PromptChatMemoryAdvisor.

error logs: [dispatcherServlet] in context with path [] threw exception [Request processing failed: software.amazon.awssdk.services.bedrockruntime.model.ValidationException: A conversation must start with a user message. Try again with a conversation that starts with a user message. (Service: BedrockRuntime, Status Code: 400, Request ID: 17f8c248-ba1d-451a-862f-2595936bec1a)] with root cause...

Environment SpringAI 1.0.0-M7 Java 17 PostgreSQL and Cassadra DB (both running in Docker) AWS Bedrock with amazon.nova-lite-v1:0 as the converse model

Steps to reproduce With the above environment, set up persistent memory with the MessageChatMemoryAdvisor. Send your first message with a chatId. Send a second message to that chatId now that the "ASSISTANT" is logged for the latest message as the "type". Error should produce.

Expected behavior I would expect MessageChatMemoryAdvisor to adhere to Bedrock's contract of a message.

Minimal Complete Reproducible example

Controller:

@PostMapping
    public String chat( 
                        @RequestParam String userMessage,
                        @RequestParam String chatId) {
        return chatService.chatNonStreaming(userMessage, chatId);
    }

Service:

    public ChatService(ChatModel chatModel, VectorStore vectorStore, ChatMemory chatMemory) {
        this.chatClient = ChatClient.builder(chatModel)
                .defaultSystem("""
                You are a support agent for a user. Respond in a friendly, helpful, and joyful manner.
            """)
                .defaultAdvisors(
                        new PromptChatMemoryAdvisor(chatMemory),
                        new QuestionAnswerAdvisor(vectorStore),
                        new SimpleLoggerAdvisor()
                )
                .build();
    }

      public String chatNonStreaming(String userMessage, String chatId) {
          ChatResponse response = this.chatClient.prompt()
                  .user(userMessage)
                  .advisors(a -> a
                          .param(CHAT_MEMORY_CONVERSATION_ID_KEY, chatId)
                          .param(CHAT_MEMORY_RETRIEVE_SIZE_KEY, 100))
                  .call()
                  .chatResponse();
  
          return response.getResult().getOutput().getText();
    }

build.gradle:

ext {
	set('springAiVersion', "1.0.0-M7")
}

dependencies {
	// AI
	implementation 'org.springframework.ai:spring-ai-starter-model-bedrock'
	implementation 'org.springframework.ai:spring-ai-starter-model-bedrock-converse'
	implementation 'org.springframework.ai:spring-ai-starter-vector-store-pgvector'
	implementation 'org.springframework.ai:spring-ai-advisors-vector-store'
	implementation 'org.springframework.ai:spring-ai-starter-model-chat-memory-jdbc'

	// AWS
	implementation platform("io.awspring.cloud:spring-cloud-aws-dependencies:3.3.0")

	// Spring Integration
	implementation 'org.springframework.integration:spring-integration-core'
	implementation("org.springframework.integration:spring-integration-aws:3.0.9")

	// web
	implementation 'org.springframework.boot:spring-boot-starter-web'
}

dependencyManagement {
	imports {
		mavenBom "org.springframework.ai:spring-ai-bom:${springAiVersion}"
	}
}

application.yml:

logging:
  level:
    org:
      springframework:
        ai:
          chat:
            client:
              advisor: DEBUG
spring:
  application:
    name: ${NAME}
  ai:
    bedrock:
      aws:
        region: us-east-1
        access-key: ${ACCESS_KEY}
        secret-key: ${SECRET_KEY}
      converse:
        chat:
          options:
            model: amazon.nova-lite-v1:0
  datasource:
    url: jdbc: ${DATABASE_URL}
    username: ${DATABASE_USERNAME}
    password: ${DATABASE_PASSWORD}

server:
  port: 8080