feat: validate API keys before execution for early feedback

[KushagraaWadhwa]

feat: Validate API Keys Before Execution for Early Feedback

🎯 Description

This PR adds early API key validation to provide users with immediate, actionable feedback when their API configuration is missing or incorrect. Instead of waiting 30+ seconds for the first LLM call to fail, users now get instant validation with helpful error messages.

✨ Changes

New Functions Added

1. validate_api_key_format(key_name: str, key_value: str) -> tuple[bool, str]

   • Validates API key format for known providers (OpenAI, Anthropic, Azure)
   • Detects common mistakes:
     • Empty keys
     • Too short keys (< 20 characters)
     • Placeholder keys (your-api-key-here, ...)
     • Incorrect prefixes (OpenAI should start with sk-, Anthropic with sk-ant-)
   • Returns tuple of (is_valid, error_message)

2. validate_api_configuration(model: str, azure_endpoint: str, llm_via_clipboard: bool) -> None

   • Main validation function called early in the execution flow
   • Intelligently determines which API key is required based on:
     • Model name (gpt-4 → OpenAI, claude → Anthropic)
     • Azure endpoint configuration
     • Local model setting
   • Provides provider-specific setup instructions with URLs
   • Skips validation for clipboard mode and local models
   • Exits with clear, colored error messages if validation fails

Integration

• Validation runs immediately after load_env_if_needed() in main() function
• Executes before creating AI instance to save time and avoid unnecessary initialization
• Uses colored terminal output for better visibility

🧪 Testing

Added comprehensive test coverage in tests/applications/cli/test_api_validation.py:

TestValidateApiKeyFormat (11 tests)

• ✅ Valid keys for OpenAI (sk-, sk-proj-), Anthropic (sk-ant-), Azure
• ✅ Empty keys detection
• ✅ Too short keys detection (< 20 chars)
• ✅ Placeholder detection (your-api-key-here, ...)
• ✅ Invalid prefix detection per provider
• ✅ Provider-specific format validation

TestValidateApiConfiguration (12 tests)

• ✅ Clipboard mode skips validation
• ✅ Local model mode skips validation
• ✅ Missing key detection for all providers (OpenAI, Anthropic, Azure)
• ✅ Valid keys pass validation
• ✅ Invalid format keys fail with proper messages
• ✅ Placeholder keys fail validation
• ✅ Model name variations (claude, anthropic/, etc.)

Total: 23 comprehensive tests with full coverage of validation logic

📊 Example Output

Before (Old Behavior)

$ gpte projects/my-project Running gpt-engineer in /projects/my-project

Loading environment... [user waits 30+ seconds] Error: openai.AuthenticationError - Invalid API key provided### After (New Behavior)

Missing API Key: $ gpte projects/my-project

Error: OPENAI_API_KEY is not set.

To use OpenAI with model 'gpt-4', you need to configure your API key:

1. Get your API key from https://platform.openai.com/api-keys
2. Set it using: export OPENAI_API_KEY='your-key'
3. Or add it to a .env file in your project directory

For more information, see: README.mdInvalid Format: $ export OPENAI_API_KEY="invalid-1234567890abcdefghijklmnop" $ gpte projects/my-project

Error: OPENAI_API_KEY should start with 'sk-' (found: 'invalid-12...')

Please check your OPENAI_API_KEY and ensure it's set correctly.

To update your API key:

1. Get your API key from https://platform.openai.com/api-keys
2. Set it using: export OPENAI_API_KEY='your-key'
3. Or add it to a .env file in your project directoryPlaceholder Key Detected: $ export OPENAI_API_KEY="your-api-key-here" $ gpte projects/my-project

Error: OPENAI_API_KEY appears to be a placeholder. Please use your actual API key.

Please check your OPENAI_API_KEY and ensure it's set correctly.Valid Key (Success): $ export OPENAI_API_KEY="sk-proj-valid1234567890abcdef" $ gpte projects/my-project Running gpt-engineer in /projects/my-project

Proceeds normally ✅## 🎁 Benefits

1. Immediate Feedback ⚡

• Users discover configuration issues in < 1 second
• No more waiting for LLM initialization and first API call to fail
• Saves time and reduces frustration

2. Better Error Messages 📝

• Clear indication of what's wrong (missing vs invalid)
• Actionable instructions on how to fix it
• Provider-specific guidance with direct URLs to get API keys

3. Common Mistake Detection 🛡️

• Catches placeholder keys from tutorials/examples
• Validates key format before making API calls
• Prevents wasted time debugging incorrect configurations

4. Smart Provider Detection 🧠

• Auto-detects provider from model name
• Handles OpenAI, Anthropic, and Azure OpenAI
• Skips validation when not needed (clipboard mode, local models)

5. Better User Experience ✨

• Colored output (red for errors, cyan for links)
• Step-by-step setup instructions
• Helpful documentation links

📝 Files Changed

Modified

• gpt_engineer/applications/cli/main.py (+141 lines)
  • Added validate_api_key_format() helper function (42 lines)
  • Added validate_api_configuration() main validation function (99 lines)
  • Integrated validation call in main() function (3 lines)

Added

• tests/applications/cli/test_api_validation.py (259 lines, new file)
  • TestValidateApiKeyFormat class with 11 test methods
  • TestValidateApiConfiguration class with 12 test methods
  • Comprehensive coverage of all validation scenarios

Total Changes: +400 lines across 2 files

🔍 Breaking Changes

None. This is purely additive functionality that improves the existing user experience without modifying any existing behavior.

💡 Edge Cases Handled

• ✅ Clipboard mode (validation skipped)
• ✅ Local model mode (validation skipped via LOCAL_MODEL env var)
• ✅ Azure OpenAI (separate API key and endpoint detection)
• ✅ Anthropic models (multiple name formats: claude, anthropic/)
• ✅ OpenAI models (default fallback)
• ✅ Various key formats (sk-, sk-proj-, sk-ant-, Bearer)

✅ Checklist

• [x] Code follows project style guidelines (black, ruff)
• [x] Added comprehensive tests (23 test cases)
• [x] All tests pass
• [x] Documentation (docstrings) added to all functions
• [x] No linting errors
• [x] Commit message follows conventional commits format
• [x] PR targets the main branch
• [x] Colored output for better UX
• [x] Handles all provider types

🧪 Testing Instructions

To test this PR locally:

Clone and setup

git checkout