AI conversation stream

[smoockpp]

Environment information

System:
  OS: Windows 11 10.0.26100
  CPU: (16) x64 AMD Ryzen 7 7700 8-Core Processor              
  Memory: 31.54 GB / 63.22 GB
Binaries:
  Node: 20.19.3 - C:\Program Files\nodejs\node.EXE
  Yarn: undefined - undefined
  npm: 10.8.2 - C:\Program Files\nodejs\npm.CMD
  pnpm: undefined - undefined
NPM Packages:
  @aws-amplify/ai-constructs: 1.5.3
  @aws-amplify/auth-construct: 1.8.1
  @aws-amplify/backend: 1.16.1
  @aws-amplify/backend-ai: 1.5.1
  @aws-amplify/backend-auth: 1.7.1
  @aws-amplify/backend-cli: 1.8.0
  @aws-amplify/backend-data: 1.6.1
  @aws-amplify/backend-deployer: 2.1.3
  @aws-amplify/backend-function: 1.14.1
  @aws-amplify/backend-output-storage: 1.3.1
  @aws-amplify/backend-secret: 1.4.0
  @aws-amplify/backend-storage: 1.4.1
  @aws-amplify/cli-core: 2.2.1
  @aws-amplify/client-config: 1.8.0
  @aws-amplify/data-construct: 1.16.3
  @aws-amplify/data-schema: 1.21.1
  @aws-amplify/deployed-backend-client: 1.8.0
  @aws-amplify/form-generator: 1.2.1
  @aws-amplify/model-generator: 1.2.0
  @aws-amplify/platform-core: 1.10.0
  @aws-amplify/plugin-types: 1.11.0
  @aws-amplify/sandbox: 2.1.2
  @aws-amplify/schema-generator: 1.4.0
  @aws-cdk/toolkit-lib: 1.1.1
  aws-amplify: 6.15.5
  aws-cdk-lib: 2.208.0
  typescript: 5.8.3
npm warn exec The following package was not found and will be installed: [email protected]

Describe the bug

Bug Report: Amplify Conversation ValidationException - Blank Text Field

🐛 Bug Description

Receiving ValidationException errors when using AWS Amplify Conversations API due to blank text fields in ContentBlock objects.

🔥 Error Details

errorType: "ValidationException"
message: "The text field in the ContentBlock object at messages.11.content.0 is blank. Add text to the text field, and try again."

🔍 Environment

• AWS Amplify Version: 6.15.5
• @aws-amplify/backend: 1.16.1
• @aws-amplify/backend-ai: 1.5.1
• Angular Version: 20.1.0
• Model: eu.anthropic.claude-sonnet-4-20250514-v1:0
• Region: EU (based on model)

📋 Steps to Reproduce

Prerequisites:

• Amplify Gen 2 project with backend-ai package
• Claude Sonnet 4 model access in EU region
• Default Bedrock quotas (2 requests/minute) which is not adjustable!

🎯 Expected Behavior

Conversation messages should be processed without ValidationException errors when text content is properly provided.

💥 Actual Behavior

ValidationException thrown indicating blank text field in ContentBlock at specific message index.

🔧 Potential Root Causes

• Secondary error caused by Bedrock throttling/rate limiting
• Issue with Amplify's internal message streaming logic
• ContentBlock serialization timing issue
• Race condition in Amplify's stream processing
• Internal validation error in AWS Bedrock conversation handling
• Error transformation when Bedrock quota limits are exceeded

📝 Code Context

Backend Configuration (Amplify Gen 2):

// conversationHandler.ts
import { handleConversationTurnEvent } from '@aws-amplify/backend-ai/conversation/runtime';

export const handler = handleConversationTurnEvent;

// data/resource.ts
export const conversationHandler = defineConversationHandlerFunction({
  entry: "./conversationHandler.ts",
  name: "conversationHandler",
  timeoutSeconds: 300,
  models: [{ modelId: 'eu.anthropic.claude-sonnet-4-20250514-v1:0' }],
});

export const myConversation = a
  .conversation({
    aiModel: {resourcePath: 'eu.anthropic.claude-sonnet-4-20250514-v1:0'},
    systemPrompt: myPrompt,
    handler: conversationHandler,
    inferenceConfiguration: { temperature: 0.3, topP: 0.9 },
    tools: [a.ai.dataTool({ name: "myTool", query: a.ref("myQuery") })]
  })
  .authorization(allow => allow.owner());

Frontend Usage (Standard Amplify API):

// 1. Create conversation
const { data: chat } = await this.client.conversations.myConversation.create();

// 2. Send message
const { errors } = await chat.sendMessage(content);

// 3. Subscribe to stream (READ-ONLY)
conversation.onStreamEvent({
  next: (event) => {
    if (event.text) {
      // We just display the text - no control over ContentBlocks
    }
  }
});

KEY POINT: We only READ from the stream. Amplify handles all ContentBlock creation/validation internally.

📊 Impact

• Severity: High - Blocks development, what about going to production!
• Frequency: Often - Triggered by Bedrock rate limits
• User Impact: Complete conversation failure, poor developer experience

🚨 Bedrock Quota Issues

This ValidationException appears to be a consequence of hitting Bedrock rate limits:

• Bedrock's ridiculously low quotas (2 requests per minute default)
• Extremely discouraging for developers trying to build applications
• Rate limiting should return proper throttling errors, not ValidationExceptions
• Current quotas make it nearly impossible to develop/test Bedrock applications
• AWS needs to provide reasonable default quotas for development

🆘 Additional Context

• Error appears at message index 11, suggesting it may be related to conversation length
• We are ONLY subscribers - we don't create or modify ContentBlocks
• Standard Amplify API usage: create() → sendMessage() → onStreamEvent()
• Using defineConversationHandlerFunction with Claude Sonnet 4
• Conversation includes data tools
• ValidationException occurs AFTER hitting Bedrock rate limits
• This is 100% an internal Amplify/Bedrock bug - we have no control over message storage
• May be related to error transformation when Bedrock throttling occurs
• Developer experience severely impacted by unrealistic quota limits

Reproduction steps

Reproduction Steps:

1. Create conversation handler:

   // conversationHandler.ts
   import { handleConversationTurnEvent } from '@aws-amplify/backend-ai/conversation/runtime';
   export const handler = handleConversationTurnEvent;
   

2. Configure conversation in data/resource.ts:

   export const conversationHandler = defineConversationHandlerFunction({
     entry: "./conversationHandler.ts",
     name: "conversationHandler",
     timeoutSeconds: 300,
     models: [{ modelId: 'eu.anthropic.claude-sonnet-4-20250514-v1:0' }],
   });
   
   export const testConversation = a.conversation({
     aiModel: {resourcePath: 'eu.anthropic.claude-sonnet-4-20250514-v1:0'},
     systemPrompt: 'You are a helpful assistant.',
     handler: conversationHandler,
     inferenceConfiguration: { temperature: 0.3, topP: 0.9 },
     tools: [a.ai.dataTool({ name: "testTool", query: a.ref("testQuery") })]
   }).authorization(allow => allow.owner());
   

3. Deploy backend:

   npx ampx sandbox
   

4. Create frontend conversation:

   const client = generateClient<Schema>();
   const { data: chat } = await client.conversations.testConversation.create();
   

5. Send multiple messages rapidly to trigger rate limiting:

   // Send 10+ messages quickly to exceed Bedrock quotas
   for (let i = 0; i < 15; i++) {
     await chat.sendMessage(`Test message ${i}`);
   }
   

6. Subscribe to stream:

   chat.onStreamEvent({
     next: (event) => console.log(event),
     error: (error) => console.error(error)
   });
   

Expected Result: Proper throttling error from Bedrock Actual Result: ValidationException about blank ContentBlock at message index 11

Key Trigger: This occurs consistently after hitting Bedrock rate limits (2 req/min default quota)