🐛 Bug: A conversation must alternate between user and assistant roles.

[ClementVanPeuter]

♻️ Reproduction Steps

1. Make a node js Endpoint with a langchain + bedrockChatConverse endpoint ( see below)
2. Git clone the todos app
3. Run both the endpoint and the todos app ( change the Provider like below).
4. Run the todos app and chat once ( " Add a task for cleaning bedroom")
5. The app first response is ok, but the endpoint crash ( see logs)

Error getting response: ValidationException: A conversation must alternate between user and assistant roles. Make sure the conversation alternates between user and assistant roles and try again.

<CopilotKit runtimeUrl="http://localhost:4000/api/copilotkit">
        <TasksProvider>
          <TasksList />
        </TasksProvider>
        <CopilotPopup />
  </CopilotKit>

The node js endpoint with langchain adapter and bedrock

import express from 'express';
import {
  CopilotRuntime,
  copilotRuntimeNodeHttpEndpoint, LangChainAdapter,
} from '@copilotkit/runtime';


const app = express();
const runtime = new CopilotRuntime();

import { ChatBedrockConverse } from "@langchain/aws";

const llm = new ChatBedrockConverse({
  model: "anthropic.claude-3-5-sonnet-20240620-v1:0",
  region: "xxxxxxx,
  credentials: {
    accessKeyId: "xxxxxxxxxxxxx",
    secretAccessKey: "xxxxxxxxxxxxxxxxx",
  },
});

const serviceAdapter = new LangChainAdapter({
  // @ts-ignore
  chainFn: async ({ messages, tools }) => {
    // @ts-ignore
    return llm.stream(messages, { tools });
  }
});

app.use('/api/copilotkit', (req, res, next) => {

  const handler = copilotRuntimeNodeHttpEndpoint({
    endpoint: '/copilotkit',
    runtime,
    serviceAdapter: serviceAdapter,
  });

  // @ts-ignore
  return handler(req, res, next);
});

app.listen(4000, () => {
  console.log('Listening at http://localhost:4000/api/copilotkit');
});

✅ Expected Behavior

CopilotKit behave normally

❌ Actual Behavior

Copilot do a firstTask but crash just after.

𝌚 CopilotKit Version

├── @copilotkit/[email protected]
├── @copilotkit/[email protected]
├── @copilotkit/[email protected]

📄 Logs (Optional)

Listening at http://localhost:4000/api/copilotkit
field[contentBlockIndex] already exists in this message chunk and value has unsupported type.
Error getting response: ValidationException: A conversation must alternate between user and assistant roles. Make sure the conversation alternates between user and assistant roles and try again.
    at de_ValidationExceptionRes (/Users/clement.vanpeuter.zenika/Desktop/redbox-copilotkit-endpoint/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1204:21)
    at de_CommandError (/Users/clement.vanpeuter.zenika/Desktop/redbox-copilotkit-endpoint/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1037:19)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /Users/clement.vanpeuter.zenika/Desktop/redbox-copilotkit-endpoint/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
    at async /Users/clement.vanpeuter.zenika/Desktop/redbox-copilotkit-endpoint/node_modules/@smithy/core/dist-cjs/index.js:165:18
    at async /Users/clement.vanpeuter.zenika/Desktop/redbox-copilotkit-endpoint/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
    at async /Users/clement.vanpeuter.zenika/Desktop/redbox-copilotkit-endpoint/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22
    at async ChatBedrockConverse._streamResponseChunks (/Users/clement.vanpeuter.zenika/Desktop/redbox-copilotkit-endpoint/node_modules/@langchain/aws/dist/chat_models.cjs:693:26)
    at async ChatBedrockConverse._streamIterator (/Users/clement.vanpeuter.zenika/Desktop/redbox-copilotkit-endpoint/node_modules/@langchain/core/dist/language_models/chat_models.cjs:94:34) {
  '$fault': 'client',
  '$metadata': {
    httpStatusCode: 400,
    requestId: 'dce3cbf4-f501-4f9d-9a13-cae3b498ff0a',
    extendedRequestId: undefined,
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  }
}
[16:27:43.849] ERROR: A conversation must alternate between user and assistant roles. Make sure the conversation alternates between user and assistant roles and try again.
    component: "Yoga GraphQL"
    err: {
      "type": "GraphQLError",
      "message": "A conversation must alternate between user and assistant roles. Make sure the conversation alternates between user and assistant roles and try again.",
      "stack":
          ValidationException: A conversation must alternate between user and assistant roles. Make sure the conversation alternates between user and assistant roles and try again.
              at de_ValidationExceptionRes (/Users/clement.vanpeuter.zenika/Desktop/redbox-copilotkit-endpoint/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1204:21)
              at de_CommandError (/Users/clement.vanpeuter.zenika/Desktop/redbox-copilotkit-endpoint/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1037:19)
              at processTicksAndRejections (node:internal/process/task_queues:95:5)
              at async /Users/clement.vanpeuter.zenika/Desktop/redbox-copilotkit-endpoint/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
              at async /Users/clement.vanpeuter.zenika/Desktop/redbox-copilotkit-endpoint/node_modules/@smithy/core/dist-cjs/index.js:165:18
              at async /Users/clement.vanpeuter.zenika/Desktop/redbox-copilotkit-endpoint/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
              at async /Users/clement.vanpeuter.zenika/Desktop/redbox-copilotkit-endpoint/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22
              at async ChatBedrockConverse._streamResponseChunks (/Users/clement.vanpeuter.zenika/Desktop/redbox-copilotkit-endpoint/node_modules/@langchain/aws/dist/chat_models.cjs:693:26)
              at async ChatBedrockConverse._streamIterator (/Users/clement.vanpeuter.zenika/Desktop/redbox-copilotkit-endpoint/node_modules/@langchain/core/dist/language_models/chat_models.cjs:94:34)
      "path": [
        "generateCopilotResponse"
      ],
      "locations": [
        {
          "line": 2,
          "column": 3
        }
      ],
      "extensions": {}
    }

[abhi7dhiru]

1. Understand the Error Message: The key part of the error message is: A conversation must alternate between user and assistant roles. 2. Examine Message Structure: Ensure that the messages being passed to the llm.stream() function adhere to the required structure. Typically, a conversation should look like this: [ { "role": "user", "content": "Add a task for cleaning bedroom" }, { "role": "assistant", "content": "Sure, I've added that task." } ] 3. Modify the Chain Function: You should confirm that the messages array being passed to the chain function correctly alternates between user and assistant roles. Here’s a suggestion on how to adjust the function: const serviceAdapter = new LangChainAdapter({ // @ts-ignore chainFn: async ({ messages, tools }) => { // Ensure the messages array starts with the user role if (messages.length > 0) { const conversation = messages.map(msg => ({ role: msg.role, content: msg.content, }));

  // You may need to ensure the format is correct here as well
  return llm.stream(conversation, { tools });
}
throw new Error("No messages provided.");

} }); 4. Implement Logging: Add logging to see the messages being sent to the model. This will help you verify that the structure is correct before sending:

javascript console.log("Messages being sent to Bedrock:", JSON.stringify(messages, null, 2)); 5. Check Incoming Requests: Ensure that the requests hitting your endpoint contain the expected data structure. You may want to log incoming requests to verify their format.

6. Update Dependencies: Make sure all your dependencies are up to date. Sometimes bugs or issues in libraries can cause unexpected behaviors. You can check for updates in your package.json and update accordingly.

7. Test with Static Data: Before testing with dynamic user input, test your endpoint with static data that you know conforms to the required structure. This will help isolate whether the issue is with user input or the overall implementation.

Example of Testing Static Data: You can modify your request handling to test with hardcoded messages to ensure the system works correctly:

javascript app.use('/api/copilotkit', (req, res, next) => { const handler = copilotRuntimeNodeHttpEndpoint({ endpoint: '/copilotkit', runtime, serviceAdapter: serviceAdapter, });

// Test with static data req.body = { messages: [ { role: "user", content: "Add a task for cleaning bedroom" }, { role: "assistant", content: "Sure, what else would you like to add?" } ] };

return handler(req, res, next); }); 8. Error Handling: Ensure that your error handling in the endpoint captures any potential issues gracefully and logs them properly.

Conclusion: By following these steps and carefully examining the message structure and flow, you should be able to resolve the validation exception. If the problem persists, consider checking the official documentation for the libraries you are using or reaching out to their support communities for further assistance.

[jeremylatorre]

1. Understand the Error Message: The key part of the error message is: A conversation must alternate between user and assistant roles. 2. Examine Message Structure: Ensure that the messages being passed to the llm.stream() function adhere to the required structure. Typically, a conversation should look like this: [ { "role": "user", "content": "Add a task for cleaning bedroom" }, { "role": "assistant", "content": "Sure, I've added that task." } ] 3. Modify the Chain Function: You should confirm that the messages array being passed to the chain function correctly alternates between user and assistant roles. Here’s a suggestion on how to adjust the function: const serviceAdapter = new LangChainAdapter({ // @ts-ignore chainFn: async ({ messages, tools }) => { // Ensure the messages array starts with the user role if (messages.length > 0) { const conversation = messages.map(msg => ({ role: msg.role, content: msg.content, }));

  // You may need to ensure the format is correct here as well
  return llm.stream(conversation, { tools });
}
throw new Error("No messages provided.");

} }); 4. Implement Logging: Add logging to see the messages being sent to the model. This will help you verify that the structure is correct before sending:

javascript console.log("Messages being sent to Bedrock:", JSON.stringify(messages, null, 2)); 5. Check Incoming Requests: Ensure that the requests hitting your endpoint contain the expected data structure. You may want to log incoming requests to verify their format.

6. Update Dependencies: Make sure all your dependencies are up to date. Sometimes bugs or issues in libraries can cause unexpected behaviors. You can check for updates in your package.json and update accordingly.

7. Test with Static Data: Before testing with dynamic user input, test your endpoint with static data that you know conforms to the required structure. This will help isolate whether the issue is with user input or the overall implementation.

Example of Testing Static Data: You can modify your request handling to test with hardcoded messages to ensure the system works correctly:

javascript app.use('/api/copilotkit', (req, res, next) => { const handler = copilotRuntimeNodeHttpEndpoint({ endpoint: '/copilotkit', runtime, serviceAdapter: serviceAdapter, });

// Test with static data req.body = { messages: [ { role: "user", content: "Add a task for cleaning bedroom" }, { role: "assistant", content: "Sure, what else would you like to add?" } ] };

return handler(req, res, next); }); 8. Error Handling: Ensure that your error handling in the endpoint captures any potential issues gracefully and logs them properly.

Conclusion: By following these steps and carefully examining the message structure and flow, you should be able to resolve the validation exception. If the problem persists, consider checking the official documentation for the libraries you are using or reaching out to their support communities for further assistance.

Thank you chatGPT

[ClementVanPeuter]

@abhi7dhiru do you really think that I haven't already tried to consult chatGPT or something else? Not helpful answer at all ( but hey I know It's hacktoberfest ! ).

[ClementVanPeuter]

Ok, so after a day of debugging, if you want to integrate Bedrock with CopilotKit, you need to do the following:

import express from 'express';
import {
  CopilotRuntime,
  copilotRuntimeNodeHttpEndpoint,
  GroqAdapter,
  LangChainAdapter,
  UnifyAdapter
} from '@copilotkit/runtime';
import { BedrockChat } from "@langchain/community/chat_models/bedrock";

const app = express();
const runtime = new CopilotRuntime();

const serviceAdapter = new LangChainAdapter({
  //@ts-ignore
  chainFn: async ({ messages, tools }) => {
    const model = new BedrockChat({
      model: "anthropic.claude-3-5-sonnet-20240620-v1:0",
      region: "xxxxxxxx",
      credentials: {
        accessKeyId: "xxxxxxxxxxxx",
        secretAccessKey: "xxxxxxxxxx",
      },
    });
    const llmWithTools = model.bindTools(tools);
    // @ts-ignore
    return llmWithTools.invoke(messages);
  }
});

app.use('/api/copilotkit', (req, res, next) => {
  const handler = copilotRuntimeNodeHttpEndpoint({
    endpoint: '/copilotkit',
    runtime,
    serviceAdapter: serviceAdapter,
  });
  // @ts-ignore
  return handler(req, res, next);
});

app.listen(4000, () => {
  console.log('Listening at http://localhost:4000/api/copilotkit');
});

You need to manually bind the tools. Please verify that the model you're using is capable of handling tools.

It's important to note that if you want to use Anthropic models, you should use BedrockChat (from LangChain) instead of ChatBedrockConverse, as it's more suitable for this purpose.

[NathanTarbert]

Hi @ClementVanPeuter, apologies for the delay. Thanks for posting the integration for Bedrock. Would you want to submit a PR?

[tylerslaton]

Going to close this issue out. If anyone needs to do this, please reference @ClementVanPeuter's excellent sleuth work 🙌🏻