Add support for custom OpenAI base URL configuration

[orestesgarcia]

Summary

This feature request adds support for configuring a custom OpenAI base URL throughout the entire Archon system, enabling users to route OpenAI API calls through proxies like LiteLLM, Azure OpenAI, or other OpenAI-compatible endpoints.

Motivation

The primary motivation for this feature is to provide flexibility in accessing the OpenAI API through various proxies and gateways:

• LiteLLM Proxy: Route requests through LiteLLM for cost optimization, rate limiting, or provider switching
• Azure OpenAI: Use Azure-hosted OpenAI models instead of direct OpenAI API
• Corporate Proxies: Support enterprise environments with required API gateways
• Local Development: Point to local OpenAI-compatible models or mock services
• Cost Management: Use services that provide better pricing or billing controls

Current State

Currently, Archon hardcodes the OpenAI API endpoint and doesn't provide a way for users to configure alternative endpoints, limiting deployment flexibility in enterprise or specialized environments.

Proposed Changes

Backend Changes

1. Credential Service Enhancement (credential_service.py)

   • Add support for OPENAI_BASE_URL setting in RAG strategy configuration
   • Enable custom base URL retrieval alongside existing API key management

2. LLM Provider Service (llm_provider_service.py)

   • Update OpenAI client creation to use custom base_url when configured
   • Maintain backwards compatibility with default OpenAI endpoint

3. Code Storage Service (code_storage_service.py)

   • Update synchronous OpenAI client to also support custom base_url
   • Ensure consistency between async and sync OpenAI usage

4. PydanticAI Agents Integration

   • Create new agent_provider_config.py module for centralized agent provider configuration
   • Update base agent class to support custom OpenAI providers
   • Modify RAG and Document agents to use configured providers
   • Update agent server to handle async provider initialization

Frontend Changes

5. RAG Settings UI (RAGSettings.tsx)
   • Add conditional input field for OpenAI Base URL (similar to Ollama base URL)
   • Show field only when OpenAI is selected as provider
   • Include helpful placeholder text

Key Features

• Optional Configuration: Base URL is optional - defaults to standard OpenAI API if not set
• UI Integration: Seamlessly integrated into existing provider selection interface
• Consistent Behavior: Works across all OpenAI usage (direct clients, PydanticAI agents, embeddings)
• Secure Storage: Settings stored encrypted in database like other credentials
• Backwards Compatible: Existing deployments continue working without changes

Use Cases

LiteLLM Integration

# Set base URL to LiteLLM proxy
OPENAI_BASE_URL=http://localhost:4000/v1

Azure OpenAI

# Point to Azure OpenAI endpoint
OPENAI_BASE_URL=https://your-resource.openai.azure.com/v1

Corporate Proxy

# Route through corporate API gateway
OPENAI_BASE_URL=https://api-gateway.company.com/openai/v1

Implementation Status

The implementation has been completed and tested:

• ✅ All OpenAI clients (async/sync) support custom base URL
• ✅ PydanticAI agents use custom providers when configured
• ✅ Frontend UI includes base URL configuration field
• ✅ Backwards compatibility maintained
• ✅ Comprehensive error handling and fallbacks

Benefits

1. Enterprise Ready: Enables deployment in corporate environments with API gateways
2. Cost Optimization: Allows use of LiteLLM or similar proxies for better pricing
3. Provider Flexibility: Switch between OpenAI, Azure, or compatible services easily
4. Development Friendly: Support for local development and testing environments
5. Future Proof: Foundation for supporting additional OpenAI-compatible providers

Files Modified

• python/src/server/services/credential_service.py
• python/src/server/services/llm_provider_service.py
• python/src/server/services/storage/code_storage_service.py
• python/src/agents/agent_provider_config.py (new)
• python/src/agents/base_agent.py
• python/src/agents/rag_agent.py
• python/src/agents/document_agent.py
• python/src/agents/server.py
• archon-ui-main/src/components/settings/RAGSettings.tsx

This feature significantly enhances Archon's flexibility and enterprise readiness while maintaining full backwards compatibility.

[coleam00]

I agree - this is a priority for us! I have a draft created in the board but I'll add this issue in place.

[wenis]

Very excited for this one. Probably a lot of us running local LLMs and would just use llama.cpp for serving up an LLM instance and embeddings.

[ShaofeiZi]

+1，looking forward to the merge!

[tazmon95]

@coleam00, want me to help with this one? I think it could reuse some of what I did for the Ollama integration.

[coleam00]

@coleam00, want me to help with this one? I think it could reuse some of what I did for the Ollama integration.

Yes I would love that John!