litellm icon indicating copy to clipboard operation
litellm copied to clipboard

[Feature]: Add support for Cohere Rerank v3.5 via Bedrock

Open taralika opened this issue 2 months ago • 2 comments

The Feature

I'd like to add Cohere Rerank v3.5 via bedrock like this:

- model_name: cohere.rerank-v3-5:0
    litellm_params:
      model: bedrock/cohere.rerank-v3-5:0
      aws_region_name: us-west-2
    model_info:
      mode: rerank

but doing so currently gives error since this model is not supported by litellm via Bedrock: {\"message\":\"The provided model ARN for reranking is invalid. Please, use a valid model ARN. Also check the model ARN follows this format: arn:<partition>:bedrock:<region>::foundation-model/<model-id>\"}

Motivation, pitch

Needed to run new cohere v3.5 reranker via bedrock

Are you a ML Ops Team?

Yes

Twitter / LinkedIn details

No response

taralika avatar Dec 06 '24 06:12 taralika