Draft / WIP: Add latest Airbyte loader (do not merge)

[aaronsteers]

This is still a work in progress...

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langflow	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 4, 2024 8:59pm

[ogabrielluiz]

Hey @aaronsteers How are you?

We moving away from the config file for 1.0. We might come back with another one until the main release but this one won't work for new components.

We could get to a solution together since Airbyte is a very powerful tool.

First thing is that all new components should be CustomComponents (we'll update the docs on this in the coming days).

We could add the loader in the src/backend/base/langflow/components/documentloaders, or, depending on the amount of dependencies it could add, we could create a generic file/data loader using airbyte and decide somewhere else to put it in.

[aaronsteers]

Hi, @ogabrielluiz! Thanks very much for this info. Once the docs and new process are updated, let me know and I'll be happy to support setting up the new integration.

[ogabrielluiz]

Ok. I can probably get it written by tomorrow.

Essentially, you'd add your component, add the dependency to the outer pyproject.toml, and test it.