litellm
litellm copied to clipboard
[Feature]: Add support for Cohere Rerank v3.5 via Bedrock
The Feature
I'd like to add Cohere Rerank v3.5 via bedrock like this:
- model_name: cohere.rerank-v3-5:0
litellm_params:
model: bedrock/cohere.rerank-v3-5:0
aws_region_name: us-west-2
model_info:
mode: rerank
but doing so currently gives error since this model is not supported by litellm via Bedrock:
{\"message\":\"The provided model ARN for reranking is invalid. Please, use a valid model ARN. Also check the model ARN follows this format: arn:<partition>:bedrock:<region>::foundation-model/<model-id>\"}
Motivation, pitch
Needed to run new cohere v3.5 reranker via bedrock
Are you a ML Ops Team?
Yes
Twitter / LinkedIn details
No response