llm icon indicating copy to clipboard operation
llm copied to clipboard

Support for AWS Bedrock?

Open swapneils opened this issue 7 months ago • 2 comments

gptel has recently been working on support for AWS Bedrock models (https://github.com/karthink/gptel/issues/379#issuecomment-2676029480). Wondering if any llm maintainers would be interested in porting it to this package?

I don't currently have bandwidth and don't use this package much myself, but it seems like it would help those who do to be able to use the variety of models Bedrock provides, especially if they want to use company-specific fine-tuned models or replicate the more professional paid LLM features in other IDEs. Few people have the compute to run non-trivial models locally, and subscription services like ChatGPT+ are often heavily overpriced for the wrappers they provide over LLMs, relative to tuning a Bedrock model and writing the tooling environment yourself in Emacs.

swapneils avatar Apr 28 '25 17:04 swapneils

This seems possible, I'll take a look. From that thread, their documentation is at https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html.

Right now there isn't a demand for this (it doesn't seem like you would use this yourself), so it's fairly low priority at the moment.

ahyatt avatar Apr 29 '25 14:04 ahyatt

Just stumbled over this request - just adding my interest here - I suspect without having dropped into the code that it may use the same foundation as with Google Cloud and their models?

Preferably project-specific (.dir-locals.el) or configured through environment variables somehow. We use nix devshells that I'd like my team (for those that use Emacs) to have automatically without having to configure it manually.

martinjlowm avatar Apr 29 '25 19:04 martinjlowm