cartography
cartography copied to clipboard
Add Airbyte intel
Summary
This PR adds a new module for Airbyte. It enables the injection of the following nodes into Cartography:
- AirbyteOrganization
- AirbyteUser
- AirbyteSource
- AirbyteDestination
- AirbyteConnection
- AirbyteWorkspace
- AirbyteTag
- AirbyteStream
Checklist
- [X] Correct all the CHANGEME in code generated by cartography-openapi
- [X] Add the necessary configuration keys in
cartography/config.py - [X] Add the required parameters in
cartography/cli.py - [X] Import the module in
cartography/sync.py - [X] Update the cartography
README.md - [X] Update the schema documentation
docs/root/usage/schema.md - [X] Full tests coverage
- [X] Include a screenshot showing what the graph looked like before and after your changes.
- [X] Include console log trace showing what happened before and after your changes.
INFO:cartography.sync:Starting sync with update tag '1748462399'
INFO:cartography.sync:Starting sync stage 'airbyte'
INFO:cartography.intel.airbyte.organizations:Loading 1 Airbyte Organizations into Neo4j.
INFO:cartography.graph.job:Finished job AirbyteOrganization
INFO:cartography.intel.airbyte.workspaces:Loading 1 Airbyte Workspaces into Neo4j.
INFO:cartography.graph.statement:Completed AirbyteWorkspace statement #1
INFO:cartography.graph.statement:Completed AirbyteWorkspace statement #2
INFO:cartography.graph.job:Finished job AirbyteWorkspace
INFO:cartography.intel.airbyte.users:Loading 1 Airbyte Users into Neo4j.
INFO:cartography.graph.statement:Completed AirbyteUser statement #1
INFO:cartography.graph.statement:Completed AirbyteUser statement #2
INFO:cartography.graph.statement:Completed AirbyteUser statement #3
INFO:cartography.graph.statement:Completed AirbyteUser statement #4
INFO:cartography.graph.statement:Completed AirbyteUser statement #5
INFO:cartography.graph.job:Finished job AirbyteUser
INFO:cartography.intel.airbyte.sources:Loading 2 Airbyte Sources into Neo4j.
INFO:cartography.graph.statement:Completed AirbyteSource statement #1
INFO:cartography.graph.statement:Completed AirbyteSource statement #2
INFO:cartography.graph.statement:Completed AirbyteSource statement #3
INFO:cartography.graph.job:Finished job AirbyteSource
INFO:cartography.intel.airbyte.destinations:Loading 1 Airbyte Destinations into Neo4j.
INFO:cartography.graph.statement:Completed AirbyteDestination statement #1
INFO:cartography.graph.statement:Completed AirbyteDestination statement #2
INFO:cartography.graph.statement:Completed AirbyteDestination statement #3
INFO:cartography.graph.job:Finished job AirbyteDestination
INFO:cartography.intel.airbyte.tags:Loading 1 Airbyte Tags into Neo4j.
INFO:cartography.graph.statement:Completed AirbyteTag statement #1
INFO:cartography.graph.statement:Completed AirbyteTag statement #2
INFO:cartography.graph.statement:Completed AirbyteTag statement #3
INFO:cartography.graph.job:Finished job AirbyteTag
INFO:cartography.intel.airbyte.connections:Loading 1 Airbyte Connections into Neo4j.
INFO:cartography.intel.airbyte.connections:Loading 10 Airbyte Streams into Neo4j.
INFO:cartography.graph.statement:Completed AirbyteStream statement #1
INFO:cartography.graph.statement:Completed AirbyteStream statement #2
INFO:cartography.graph.statement:Completed AirbyteStream statement #3
INFO:cartography.graph.job:Finished job AirbyteStream
INFO:cartography.graph.statement:Completed AirbyteConnection statement #1
INFO:cartography.graph.statement:Completed AirbyteConnection statement #2
INFO:cartography.graph.statement:Completed AirbyteConnection statement #3
INFO:cartography.graph.statement:Completed AirbyteConnection statement #4
INFO:cartography.graph.statement:Completed AirbyteConnection statement #5
INFO:cartography.graph.statement:Completed AirbyteConnection statement #6
INFO:cartography.graph.job:Finished job AirbyteConnection
INFO:cartography.sync:Finishing sync stage 'airbyte'
INFO:cartography.sync:Finishing sync with update tag '1748462399'