crawl4ai icon indicating copy to clipboard operation
crawl4ai copied to clipboard

feat: Add Azure OpenAI configuration support to crwl config

Open arpagon opened this issue 4 months ago • 0 comments

Summary

This PR adds support for configuring Azure OpenAI settings through crwl config, eliminating the need to manually set environment variables before each command.

Changes

  • Added AZURE_API_BASE configuration setting to USER_SETTINGS in config.py
  • Added AZURE_API_VERSION configuration setting to USER_SETTINGS in config.py
  • Updated setup_llm_config() in cli.py to export Azure environment variables from the global configuration

Motivation

Currently, when using Azure OpenAI with crawl4ai, users must export AZURE_API_BASE and AZURE_API_VERSION environment variables before running each command:

AZURE_API_BASE="https://your-resource.openai.azure.com" \
AZURE_API_VERSION="2024-08-01-preview" \
crwl https://example.com -q "What is this about?"

This is inconvenient and error-prone. With this PR, users can configure these settings once:

crwl config set AZURE_API_BASE "https://your-resource.openai.azure.com"
crwl config set AZURE_API_VERSION "2024-08-01-preview"
crwl config set DEFAULT_LLM_PROVIDER "azure/gpt-5"

And then use crawl4ai with Azure OpenAI without any environment variables:

crwl https://example.com -q "What is this about?"

Testing

Tested locally with Azure OpenAI GPT-5 deployment:

  • Configured Azure settings via crwl config set
  • Verified settings are stored in ~/.crawl4ai/global.yml
  • Successfully ran crwl commands without setting environment variables
  • Confirmed LiteLLM received the Azure environment variables correctly

Implementation Details

The setup_llm_config() function now reads AZURE_API_BASE and AZURE_API_VERSION from the global configuration and exports them as environment variables before LiteLLM is called. This ensures backward compatibility while providing a better user experience.

🤖 Generated with Claude Code

Co-Authored-By: Claude [email protected]

arpagon avatar Nov 02 '25 17:11 arpagon