LLM-Support Azure

[rwang1987]

Please provide examples of how to use MCP agent with Azure hosted models like OpenAI models.

[saqadri]

@rwang1987 thank you for asking! It's very similar to the Ollama example: https://github.com/lastmile-ai/mcp-agent/blob/main/examples/mcp_basic_ollama_agent/mcp_agent.config.yaml#L24-L25

Simply specify the base_url to be your Azure model deployment URL. Should be something like https://${azure_resource}.openai.azure.com/openai/deployments/{model_name}, and specify api_key to be the corresponding API key.

Everything else stays exactly the same.

Please let me know if you run into any issues! Also, if you get it working, please contribute an example to the repo for others to benefit!

[rwang1987]

@saqadri thank you for your prompt reply! However I got no luck using the ollama as an example to get it working with our Azure OpenAI endpoints.

Here is example code of using Azure AI Inference with the same credentials that's working with our Azure endpoints:

from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import SystemMessage, UserMessage
from azure.core.credentials import AzureKeyCredential

endpoint = "https://agentic-ai-hub-dev-e2-openai.openai.azure.com/openai/deployments/gpt-4o"
model_name = "gpt-4o"

client = ChatCompletionsClient(
    endpoint=endpoint,
    credential=AzureKeyCredential("<API KEY>"),
)

response = client.complete(
    messages=[
        SystemMessage(content="You are a helpful assistant."),
        UserMessage(content="I am going to Paris, what should I see?")
    ],
    max_tokens=4096,
    temperature=1.0,
    top_p=1.0,
    model=model_name
)

print(response.choices[0].message.content)

And when using the Ollama example code, here is the mcp_agent.config.yaml using the same credential:

$schema: ../../schema/mcp-agent.config.schema.json

execution_engine: asyncio
logger:
  type: console
  level: debug
  batch_size: 100
  flush_interval: 2
  max_queue_size: 2048
  http_endpoint:
  http_headers:
  http_timeout: 5

mcp:
  servers:
    fetch:
      command: "uvx"
      args: ["mcp-server-fetch"]
    filesystem:
      command: "npx"
      args: ["-y", "@modelcontextprotocol/server-filesystem"]

openai:
  base_url: "https://agentic-ai-hub-dev-e2-openai.openai.azure.com/openai/deployments/gpt-4o"
  api_key: "<API KEY>"

And here is the error I got:

    "message": "Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}",
    "body": {
      "code": "404",
      "message": "Resource not found"
    },
    "code": "404",
    "param": null,
    "type": null,
    "response": "<httpx.Response [404] https://agentic-ai-hub-dev-e2-openai.openai.azure.com/openai/deployments/gpt-4o/chat/completions>",
    "status_code": 404,
    "request_id": null
  }
}
[ERROR] 2025-03-26T14:45:55 mcp_agent.workflows.llm.augmented_llm_openai.finder - Error: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
[DEBUG] 2025-03-26T14:45:55 mcp_agent.workflows.llm.augmented_llm_openai.finder - Chat finished
{
  "data": {
    "progress_action": "Finished",
    "model": "gpt-4o",
    "agent_name": "finder"
  }
}

[StreetLamb]

I’m going to close this issue now that we’ve added support for Azure models. For an example of how to run them, please refer to examples/mcp_basic_azure_agent.