Archon icon indicating copy to clipboard operation
Archon copied to clipboard
verified publisher icon coleam00

Feature/self hosted supabase

Open GQAdonis opened this issue 3 months ago • 8 comments

Pull Request

Summary

This PR makes it possible to use Archon with a self-hosted Supabase installation as well as the hosted version.

Changes Made

  • Made sure that self-hosted supabase installations are supported
  • Fixed URL path issues related to supporting this in production using Nginx proxies and SSL (credentialsService.ts extra /api in the URL path used.

Type of Change

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • [ ] Documentation update
  • [ ] Performance improvement
  • [ ] Code refactoring

Affected Services

  • [ ] Frontend (React UI)
  • [ ] Server (FastAPI backend)
  • [ ] MCP Server (Model Context Protocol)
  • [ ] Agents (PydanticAI service)
  • [ ] Database (migrations/schema)
  • [ ] Docker/Infrastructure
  • [ ] Documentation site

Testing

  • [ ] All existing tests pass
  • [ ] Added new tests for new functionality
  • [ ] Manually tested affected user flows
  • [ ] Docker builds succeed for all services

Test Evidence

# Example: python -m pytest tests/
# Example: cd archon-ui-main && npm run test

Checklist

  • [ ] My code follows the service architecture patterns
  • [ ] If using an AI coding assistant, I used the CLAUDE.md rules
  • [ ] I have added tests that prove my fix/feature works
  • [ ] All new and existing tests pass locally
  • [ ] My changes generate no new warnings
  • [ ] I have updated relevant documentation
  • [ ] I have verified no regressions in existing features

Breaking Changes

Additional Notes

GQAdonis avatar Sep 18 '25 21:09 GQAdonis

@GQAdonis , I don't understand why you need this. I've used self-hosted Supabase since the beginning with no change required.

tazmon95 avatar Sep 18 '25 22:09 tazmon95

Thanks for your reply! The scenario I was unable to get to work was this:

  • I had a self-hosted installation of Supabase on the same VM server as my Archon installation.
  • This meant the best way to reference the server in the configuration was to use a non-HTTPS URL (e.g., http://supabase-kong:8000) after connecting Archon to the Docker network for my Supabase installation. I got errors stating that unless I was using localhost as the DNS name, the Supabase server I referenced HAD to be HTTPS.

I had the following .env for my Archon installation:

# Production environment configuration
SUPABASE_URL=http://supabase-kong:8000
SUPABASE_SERVICE_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoic2VydmljZV9yb2xlIiwiaXNzIjoicHJvbWV0aGV1c2Fncy5haSIsInN1YiI6InRqYW1lc0Bwcm9tZXRoZXVzYWdzLmFpIiwiaWF0IjoxNjQxNzY5MjAwLCJleHAiOjE3OTk1MzU2MDB9.J_kSi-8wzNWLbvQJ8Vz6n_5rOFUFLX6epqRF_pfd3ZA
OPENAI_API_KEY=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

# Service Ports Configuration
ARCHON_SERVER_PORT=8181
ARCHON_MCP_PORT=8051
ARCHON_AGENTS_PORT=8052
AGENTS_ENABLED=true

# Production host configuration
HOST=archon.prometheusags.ai
# Don't set VITE_API_URL - let frontend use relative URLs that nginx will proxy
# Frontend will use /api/* paths, nginx will proxy to localhost:8181
VITE_ALLOWED_HOSTS=archon.prometheusags.ai,archon-api.prometheusags.ai
VITE_SHOW_DEVTOOLS=true

# Enable production mode for proper API URL handling
PROD=true

# Docker environment flag
DOCKER_ENV=true

With the codebase as is, this would never work, because the Archon server service would never start up due to the error. Let me know if I missed something here, but it doesn't make sense in my case for the Archon server to have to go outside the current network and back in so that it can reference https://supabase.prometheusags.ai and incur the HTTPS overhead and latency when the Supabase Kong service is available securely via HTTP on the same box.

GQAdonis avatar Sep 19 '25 20:09 GQAdonis

@GQAdonis - what if you used host.docker.internal instead of supabase-kong? That is what works for me right now!

coleam00 avatar Sep 20 '25 18:09 coleam00

The problems with using host.docker.internal can be the following:

  1. The server you talk to must expose an external port accessible from the rest of the machine and you may not want that or you may be wanting to avoid port conflicts for a service that is never used by anything but your app.

  2. You incur latency talking from the source container through the Docker bridge to the host network stack to the host loopback then back to the docker bridge then to the target container.

It's maybe not that big of a deal but it was easy to fix.

GQAdonis avatar Sep 20 '25 18:09 GQAdonis

@GQAdonis Agree, easy to fix but from a product standpoint I dont think this is something we should do. Compared to using supabase cloud, I dont think there is also no big difference in using host.docker.internal or the docker network.

As we maybe want to switch the db or support others in the future the solution is not generic in my view.

leex279 avatar Sep 20 '25 20:09 leex279

@GQAdonis, Can you just use the IP?

This is how I run my Archon with a local network Supabase. It works for me both on the same host or other hosts because of the updated made for local network ranges in this PR https://github.com/coleam00/Archon/pull/417.

image ****

I do think we should look at making Archon work better with the Supabase Docker Network if installed on the same host.

tazmon95 avatar Sep 20 '25 21:09 tazmon95

This PR, interests me so I had Claude build up a full review on it.

Thorough Analysis of PR #704: Self-Hosted Supabase Support

  Executive Summary

  PR #704 introduces comprehensive support for self-hosted Supabase deployments in Archon. The changes focus on three key areas: frontend API routing flexibility, Docker Compose networking integration, and enhanced URL 
  validation for containerized environments. This is a significant infrastructure improvement that enables Archon to work seamlessly with both cloud and self-hosted Supabase instances.

  Core Changes Analysis

  1. Frontend API Configuration Overhaul (archon-ui-main/src/config/api.ts)

  Key Transformation: Complete rewrite of API URL determination logic from simple proxy-based routing to intelligent environment-aware URL construction.

  Critical Changes:
  - Production Domain Detection: Adds specific logic for prometheusags.ai domain detection
  - HTTPS Protocol Enforcement: Automatically converts HTTP to HTTPS for production environments
  - Port Management: Removes non-standard ports (8181, 80) when converting to HTTPS for production domains
  - Proxy vs Direct Routing: Distinguishes between proxy setups and direct backend connections

  Impact: Enables frontend to work correctly with both nginx-proxied production deployments and direct self-hosted Supabase connections.

  2. Service URL Pattern Changes (Multiple Files)

  Systematic Change: Removal of /api prefix from service calls across all frontend services.

  Files Modified:
  - agentChatService.ts: 7 endpoints changed from /api/agent-chat/* to /agent-chat/*
  - credentialsService.ts: 8 endpoints changed from /api/credentials/* to /credentials/*
  - ollamaService.ts: 6 endpoints changed from /api/ollama/* to /ollama/*
  - bugReportService.ts: Changed from /api/bug-report/* to /bug-report/*
  - FeaturesSection.tsx: Changed from /api/projects/health to /projects/health

  Rationale: This change suggests the backend is now handling /api routing internally, allowing for more flexible proxy configurations.

  3. Docker Compose Networking Enhancement

  Major Addition: Integration with external supabase_default network.

  Changes:
  - Added env_file: - .env to all three services (archon-server, archon-mcp, archon-agents)
  - Added supabase_default external network to all services
  - Maintains existing app-network for internal communication

  Purpose: Enables Archon containers to communicate directly with self-hosted Supabase containers running in the same Docker environment.

  4. Enhanced Supabase URL Validation (python/src/server/config/config.py)

  Comprehensive Security Update: Expanded URL validation to support containerized environments while maintaining security.

  New Features:
  - Container Detection: Multiple methods to detect containerized environments
  - Docker/Kubernetes Patterns: Recognition of service naming patterns
  - Permissive Container Mode: More flexible validation when running in containers
  - Enhanced Local Host Detection: Support for additional local development patterns

  Security Maintained: Still blocks external domains in non-containerized environments.

  Technical Architecture Impact

  Networking Topology Changes

  Before PR #704:
  Frontend → Vite Proxy → Backend (port 8181) → Cloud Supabase

  After PR #704:
  Option 1 (Cloud): Frontend → Nginx Proxy → Backend → Cloud Supabase
  Option 2 (Self-hosted): Frontend → Direct/Proxy → Backend → Self-hosted Supabase (same network)

  URL Resolution Logic

  The new getApiUrl() function implements sophisticated logic:

  1. Environment Variable Priority: VITE_API_URL takes precedence
  2. Production Detection: Identifies production environments through multiple signals
  3. Protocol Security: Enforces HTTPS for production domains
  4. Port Normalization: Removes non-standard ports for HTTPS production URLs

  Deployment Compatibility

  Cloud Deployments (No Breaking Changes)

  - Existing cloud Supabase setups continue working
  - nginx proxy configurations remain compatible
  - HTTPS enforcement improves security

  Self-Hosted Deployments (New Capability)

  - Docker Compose can include Supabase services
  - Network isolation between Archon and Supabase components
  - Local development with self-hosted Supabase supported

  Hybrid Deployments (Enhanced Flexibility)

  - Mix of cloud services (e.g., authentication) with self-hosted database
  - Custom proxy configurations supported
  - Development-to-production migration simplified

  Security Considerations

  Positive Security Enhancements:

  - HTTPS Enforcement: Production domains automatically use HTTPS
  - Domain Validation: External domains still require HTTPS in non-containerized environments
  - Container Detection: Smart detection prevents overly permissive validation outside containers

  Potential Security Vectors:

  - Expanded HTTP Tolerance: More permissive HTTP acceptance in containerized environments
  - Pattern-Based Validation: Service name patterns could potentially be spoofed

  Overall Assessment: Security improvements outweigh risks due to HTTPS enforcement and smart container detection.

  Migration Path

  For Existing Users:

  1. No Action Required: Existing cloud setups continue working
  2. Optional: Set VITE_API_URL for custom routing
  3. Optional: Enable self-hosted Supabase with Docker Compose network integration

  For New Self-Hosted Users:

  1. Required: Configure Docker network connectivity
  2. Required: Set appropriate Supabase URLs in environment variables
  3. Optional: Customize proxy settings for production deployment

  Risk Assessment

  Low Risk Items:

  - ✅ Frontend service URL changes (systematic and tested)
  - ✅ Docker network additions (additive only)
  - ✅ HTTPS enforcement (security improvement)

  Medium Risk Items:

  - ⚠️ Complex API URL logic (multiple code paths)
  - ⚠️ Container detection methods (environment-dependent)
  - ⚠️ Backward compatibility with custom configurations

  High Risk Items:

  - � None identified - changes are largely additive and maintain backward compatibility

  Testing Recommendations

  Critical Test Scenarios:

  1. Cloud Supabase + Development Environment (existing workflow)
  2. Cloud Supabase + Production nginx proxy (existing production)
  3. Self-hosted Supabase + Docker Compose (new capability)
  4. Mixed environment with custom VITE_API_URL (edge case)

  Validation Points:

  - All API endpoints resolve correctly
  - HTTPS enforcement works properly
  - Docker network connectivity established
  - Environment variable precedence respected

  Recommendation

  APPROVE - This PR represents a well-architected solution for self-hosted Supabase support that maintains backward compatibility while adding significant new capabilities. The changes are systematic, security-conscious,
   and follow established patterns. The risk profile is low due to the additive nature of changes and comprehensive validation improvements.

  Deployment Strategy: Recommend gradual rollout with monitoring of API routing and network connectivity in production environments.

tazmon95 avatar Sep 20 '25 21:09 tazmon95

Thanks, guys! I am definitely willing to roll with the group on these things for sure. Excited to contribute!

GQAdonis avatar Sep 21 '25 14:09 GQAdonis