gorilla icon indicating copy to clipboard operation
gorilla copied to clipboard

Support for watsonx.ai inference

Open pawelknes opened this issue 7 months ago • 0 comments

Added WatsonxAIHandler which allows performing inference using the watsonx.ai platform

To authenticate access to the watsonx.ai, users need to set WX_AI_API_KEY, WX_AI_URL, and WX_AI_PROJECT_ID or WX_AI_SPACE_ID env variables.

Added support for 8 FC models provided by IBM:

  • 3 Granite models (3.1, 3.2, 3.3 8b Instruct)
  • 2 Llama models (Llama 4 Maverick and Llama 3.3 70b Instruct)
  • 3 Mistral AI models (Large 2, Medium 3, Small 3.1 24b Instruct)

pawelknes avatar May 15 '25 15:05 pawelknes