PyAirbyte
PyAirbyte copied to clipboard
Refactor: new "Writers" interface for destinations and caches
Summary by CodeRabbit
-
New Features
- Enhanced stream management with new methods for primary key and cursor key retrieval, as well as improved write strategy resolution in the
CatalogProvider
. - Streamlined processing of Airbyte messages in the
SqlProcessorBase
, supporting various message types and improving state management. - Expanded telemetry capabilities to accommodate a broader range of destination types, including support for
AirbyteWriterInterface
. - Introduced
AirbyteWriterInterface
for flexible message writing across various integrations. - Added the
_write_airbyte_message_stream
method toCacheBase
to facilitate reading and writing of Airbyte messages.
- Enhanced stream management with new methods for primary key and cursor key retrieval, as well as improved write strategy resolution in the
-
Bug Fixes
- Improved clarity in the
Destination
class'swrite
method documentation and handling of write strategies.
- Improved clarity in the
-
Documentation
- Updated method docstrings to clarify functionality and expected behavior for enhanced user understanding.
-
Refactor
- Restructured the
FileWriterBase
class to align with theAirbyteWriterInterface
, enhancing its integration within the Airbyte framework.
- Restructured the