feat: Support connector_builder_project_id in get_custom_source_definition

[aaronsteers]

feat: Support connector_builder_project_id in get_custom_source_definition

⚠️ Blocked By

This PR is blocked by https://github.com/airbytehq/airbyte-platform-internal/pull/18082 which investigates API endpoint improvements needed for the reverse lookup functionality (builder_project_id → definition_id).

The current implementation uses /connector_builder_projects/get_with_manifest but this endpoint returns sourceDefinitionId: null for published YAML connectors, preventing the intended functionality from working reliably.

---

Summary

Modified CloudWorkspace.get_custom_source_definition() to accept either definition_id OR connector_builder_project_id as lookup parameters, with validation to ensure exactly one is provided.

Key changes:

• Changed method signature to use keyword-only parameters (breaking change)
• Added connector_builder_project_id parameter (yaml-only)
• Added parameter validation (mutually exclusive, yaml-only for builder_project_id)
• Implemented lookup by builder_project_id via API endpoint
• Updated existing test to use keyword arguments
• Added unit test for parameter validation
• Added integration test for builder_project_id lookup path

Implementation note: Uses /connector_builder_projects/get_with_manifest API endpoint to resolve builder_project_id to definition_id. However, this endpoint currently returns null for published connectors (see blocker above).

Review & Testing Checklist for Human

⚠️ Risk Level: YELLOW - Breaking API change with API dependency

• [ ] Breaking change: Verify no existing code calls this method with positional arguments (search codebase for get_custom_source_definition()
• [ ] Validation logic: Test that the XOR validation correctly rejects both parameters missing, both parameters provided, and builder_project_id with non-yaml type
• [ ] Lookup logic: Test the builder_project_id lookup path with real cloud workspace data to verify it correctly finds matching definitions (currently blocked by API limitation)
• [ ] Error handling: Test the error case when builder_project_id doesn't match any definition
• [ ] API dependency: Confirm whether the API endpoint fix in the blocker PR resolves the reverse lookup issue

Recommended Test Plan

1. Call method with definition_id only (should work as before)
2. Call method with connector_builder_project_id only for a yaml definition (currently may fail due to API returning null)
3. Call method with both parameters (should raise PyAirbyteInputError)
4. Call method with neither parameter (should raise PyAirbyteInputError)
5. Call method with connector_builder_project_id for docker type (should raise PyAirbyteInputError)
6. Call method with non-existent connector_builder_project_id (should raise AirbyteError)

Notes

• Requested by: AJ Steers (@aaronsteers)
• Devin session: https://app.devin.ai/sessions/6687e8819f414215848acb29983ff644
• The builder_project_id lookup uses direct API endpoint but encounters API limitation
• All lint and type checks pass locally

Summary by CodeRabbit

• New Features

  • Support fetching custom YAML source definitions via a connector-builder project ID in addition to direct definition ID.
  • Added a deletion API for custom source definitions (YAML path).

• Bug Fixes

  • Enhanced input validation: require exactly one identifier, disallow builder-project lookups for non-YAML types, with clearer error messages and Docker path marked unsupported.

• Tests

  • Added integration tests for validation scenarios and connector-builder project ID retrieval.

[!IMPORTANT] Auto-merge enabled.

This PR is set to merge automatically when all requirements are met.