ax
ax copied to clipboard
ax-llm
Aws bedrock
-
I'm submitting a ... [ ] bug report [X ] feature request [ ] question about the decisions made in the repository [ ] question about how to use this project
-
Summary
Looking to add custom providers but immediately looking for AWS Bedrock provider. Possible to add that?
This helps to integrate with new providers easily. Looking forward for an update soon.
I can see a PR for this, but can't seem to find documentation. Any info about how to use Bedrock at this point would be great
Thanks for making Ax. Love the library so far. Was curious if you guys were planning on integrating Bedrock? We'd be happy to support
Totally open to supporting Bedrock last I checked it required some AWS specific libraries etc. If this is still the case we can add it as a separate package in the monorepo. If it's plain old HTTP then we can include it in the core ax package.
@dosco forked the codebase and got a AxAIBedrockImpl working. The OSS 120B/20B models are giving me some headaches failing to return structured responses but the bedrock claude sonnet 4 seems to work just fine. Happy to create a PR and have you review/finalize it
@dosco forked the codebase and got a AxAIBedrockImpl working. The OSS 120B/20B models are giving me some headaches failing to return structured responses but the bedrock claude sonnet 4 seems to work just fine. Happy to create a PR and have you review/finalize it
yes that would be awesome. it's not dependant on any external packages right?
@dosco you need to bring in two classes from aws sdk.
import { BedrockRuntimeClient, InvokeModelCommand } from '@aws-sdk/client-bedrock-runtime';
probably better in a separate package in the multi-repo src/ax-ai-aws-bedrock.
aws-bedrock support released. https://www.npmjs.com/package/@ax-llm/ax-ai-aws-bedrock we might need to add it to the docs