[BUG] Model Compatibility Issues with Claude Chat Bedrock API

[caosinf777]

GitHub Issue: Model Compatibility Issues with Claude Chat Bedrock API

Description

Issue Summary

We've identified a critical issue with the Claude Chat Bedrock API where certain models successfully process initial requests but fail to persist conversations, resulting in "No conversation found with id" errors when attempting to retrieve conversation history.

Environment

• Backend: Node.js/Express with AWS SDK
• Database: DynamoDB
• Models tested:
  • ✅ Working: claude-v3-haiku, claude-v3.5-sonnet, amazon-nova-pro
  • ❌ Not working: claude-v3.5-sonnet-v2, claude-v3-opus, claude-v3.5-haiku, and others

Steps to Reproduce

1. Send a request to the API with a non-working model (e.g., claude-v3.5-sonnet-v2)
2. Receive a successful response with a valid conversationId and messageId
3. Attempt to retrieve the conversation using the provided conversationId
4. Receive a 404 error with message: {"errors": ["No conversation found with id: {conversationId}"]}

Expected Behavior

All models listed in the application configuration should successfully process requests AND persist conversations that can be retrieved later using the provided conversation ID.

Actual Behavior

While all models successfully process the initial request and return a response with conversation and message IDs, only specific models (claude-v3-haiku, claude-v3.5-sonnet, amazon-nova-pro) correctly persist conversations that can be retrieved later.

Code References

The API validates model input with the following allowed values:

"Input should be 'claude-instant-v1', 'claude-v2', 'claude-v3-sonnet', 'claude-v3.5-sonnet', 'claude-v3.5-sonnet-v2', 'claude-v3.5-haiku', 'claude-v3-haiku', 'claude-v3-opus', 'mistral-7b-instruct', 'mixtral-8x7b-instruct', 'mistral-large', 'amazon-nova-pro', 'amazon-nova-lite' or 'amazon-nova-micro'"

However, when using models like claude-v3.5-sonnet-v2, the conversation persistence fails.

Example Request/Response

Request:

{
  "message": {
    "content": [
      {
        "contentType": "text",
        "body": "What are the best practices for implementing security in AWS applications?"
      }
    ],
    "model": "claude-v3.5-sonnet-v2"
  },
  "continueGenerate": false
}

Response:

{
  "conversationId": "01JN547EXA1SEV1XS23FHZKKJT",
  "messageId": "01JN547EXARW70Z5N17RQ9FGW3"
}

When retrieving the conversation:

{
  "errors": [
    "No conversation found with id: 01JN547EXA1SEV1XS23FHZKKJT"
  ]
}

Potential Causes

Based on our investigation, we believe the issue may be related to one or more of the following:

1. Response format differences: Some models may return responses in a format that our persistence layer doesn't handle correctly.

2. API incompatibility: Different models may require different API endpoints or parameters.

3. Persistence layer implementation: The code that saves conversations to the database may have model-specific handling that doesn't work for all models.

Diagnostic Information

When using a working model like claude-v3-haiku, we can successfully:

1. Send a message
2. Receive a response with conversation ID
3. Retrieve the conversation history with all message details

When using a non-working model like claude-v3.5-sonnet-v2:

1. Send a message
2. Receive a response with conversation ID
3. When trying to retrieve the conversation: {"errors": ["No conversation found with id: 01JN547EXA1SEV1XS23FHZKKJT"]}

Questions for Investigation

1. Is there a difference in how the persistence layer handles responses from different models?
2. Are we using the correct API endpoints for all models?
3. Do newer models require different parameters or handling in the conversation repository?

[statefb]

Could you provide a log? Is there any error?

[statefb]

#862 merged. Please check if the problem resolved. Thank you

[github-actions[bot]]

This issue has been labeled as "stale" due to no response by the reporter within 1 month (and 14 days after last commented by someone). And it will be closed automatically 14 days later if not responded.

[github-actions[bot]]

This issue has been closed due to no response within 14 days after labeled as "stale", 14 days after last reopened, and 14 days after last commented.