Feature Request: Ability to load custom connectors built using no code builder

[bindipankhudi]

To be explored: We might have to point to a connector python package or load a yaml file.

[natikgadzhi]

@aaronsteers I took a quick look! airbyte-cdk has sources.declarative.ManifestDeclarativeSource which loads a yaml file and provides an Airbyte CDK Source implementation on top of it.

It's used in connector builder CDK wrapper here.

The easiest way to add this to PyAirbyte would be:

1. Extract an abstract class interface from pyairbyte.Source
2. Add a PyAirbyte DeclarativeSource that implements PyAirbyte.Source, but internally wraps the Declarative Source from the CDK instead of managing pip dependency and shelling out into it.

I think I have everything I'd need to hack on it and make a nice tutorial, but time. 🙃

[ifokeev]

Need this feature a lot. Airbyte has a very nice no-code builder, but launching jobs from airbyte itself is very slow. Pyairbyte could be a good replacement

[aaronsteers]

Related work:

• https://github.com/airbytehq/airbyte/pull/36501