🚀 Feature Request: Amazon Bedrock Service Adapter

[dreamorosi]

🎤 Tell us your idea

It would be great if we could use CopilotKit with Amazon Bedrock as LLM backend.

Me & my team would like to propose and contribute the implementation of a Bedrock LLM Service Adapter similar to the ones here that you have for other providers.

This would be useful to any user who wants to use CopilotKit with Bedrock on AWS.

If the idea is welcome, we'd like to propose a PR with the implementation in the coming days.

[linear[bot]]

CPK-2430 🚀 Feature Request: Amazon Bedrock Service Adapter

[NathanTarbert]

Hey @dreamorosi, you should be able to use the LangChain adapter for Amazon Bedrock.

Check out the CopilotKit docs: https://docs.copilotkit.ai/reference/classes/llm-adapters/LangChainAdapter LangChain docs: https://python.langchain.com/docs/integrations/llms/bedrock/

Let me know if this works for you.

[dreamorosi]

Hi @NathanTarbert, yes the LangChain adapter can work, however I'd rather have an out of the box provider similar to the Anthropic, Google, OpenAI, etc.

Some of the providers like the Google one, are literally just an extended LangChainAdapter:

export class GoogleGenerativeAIAdapter extends LangChainAdapter {
  constructor(options?: GoogleGenerativeAIAdapterOptions) {
    super({
      chainFn: async ({ messages, tools, threadId }) => {
        const model = new ChatGoogle({
          modelName: options?.model ?? "gemini-1.5-pro",
          apiVersion: "v1beta",
        }).bindTools(tools);
        return model.stream(messages, { metadata: { conversation_id: threadId } });
      },
    });
  }
}

So I was hoping we could have one for Bedrock as well.

I don't have a strong opinion on whether it should be something like the Google one extending the LangChainAdapter or one that builds on the AWS SDK like others do (i.e. Groq or Anthropic). I'm happy to implement it either way based on maintainers input.

[tylerslaton]

Hey @dreamorosi, we would be happy to review and ultimately merge a contribution here.

I would suggest making this adapter with the LangChain extension like you showed. However, if you do notice some limitation there we would also be happy to have a more "from the ground up" solution like Groq or Anthropic.

We're happy to provide developer resources on our side as well to get your going here.

Thank you in advance!

[dreamorosi]

Amazing, me and my team will work on it and have a PR. We can move on from there and iterate

[NathanTarbert]

Great, @dreamorosi, thanks for the update.

Would you be open to a short meeting so we can understand if you need any guidance from the team?

[dreamorosi]

Hi @NathanTarbert - thanks for the offer, always happy to connect with other maintainers.

Give us a few days to get familiar with the codebase, and maybe we can have a call sometime early next week.

This way we can come with some informed question/doubt we might have.

What do you think?

[NathanTarbert]

Hey @dreamorosi, sure thing, that works great.

I will just add my calendar link here, so please feel free to book at a time when it's convenient for you. https://cal.com/nathan-tarbert-copilotkit/30min

[dreamorosi]

Hi @NathanTarbert, we opened a PR and have the adapter working for the most part. We chose to extend the LangChain adapter similar to what was done with the Google adapter to keep things simple and not duplicate logic that is already being handled by the framework.

We are seeing some transient errors in some of the examples due to how messages are sent and Amazon Bedrock's requirement of having messages with role user as first ones in the conversation and I think it's time we take up your offer of getting together in a call to discuss this and look at the PR.

I'll look at your calendar a book a slot.

[Siddhesh-Shinde]

Any updates on this?

[dreamorosi]

The PR was merged #1977 (original #1810) and according to the maintainers it will be included in the next release!

[NathanTarbert]

Thank you for an amazing contribution @dreamorosi!

[Siddhesh-Shinde]

Docs are needed to be updated as well