llama-stack
llama-stack copied to clipboard
fix broken bedrock inference provider
Support for Bedrock inference providers has been applied with the following merges:
https://github.com/meta-llama/llama-stack/commit/95abbf576b4b078e72b779f534cbaf696e30ecab
However, it was overwritten in the next merge.
https://github.com/meta-llama/llama-stack/commit/56aed59eb4c9915676c6fc7aac009dad97e7ead2
As a result, Bedrock is not displayed as an inference provider.
llama stack list-providers inference
+------------------------+----------------------------------------------------------------------------------+
| Provider Type | PIP Package Dependencies |
+------------------------+----------------------------------------------------------------------------------+
| remote | |
+------------------------+----------------------------------------------------------------------------------+
| meta-reference | accelerate,blobfile,fairscale,fbgemm- |
| | gpu==0.8.0,torch,torchvision,transformers,zmq |
+------------------------+----------------------------------------------------------------------------------+
| remote::sample | |
+------------------------+----------------------------------------------------------------------------------+
| remote::ollama | ollama |
+------------------------+----------------------------------------------------------------------------------+
| remote::tgi | huggingface_hub,aiohttp |
+------------------------+----------------------------------------------------------------------------------+
| remote::hf::serverless | huggingface_hub,aiohttp |
+------------------------+----------------------------------------------------------------------------------+
| remote::hf::endpoint | huggingface_hub,aiohttp |
+------------------------+----------------------------------------------------------------------------------+
| remote::fireworks | fireworks-ai |
+------------------------+----------------------------------------------------------------------------------+
| remote::together | together |
+------------------------+----------------------------------------------------------------------------------+
I fix it.
+------------------------+----------------------------------------------------------------------------------+
| remote::bedrock | boto3 |
+------------------------+----------------------------------------------------------------------------------+