OpenMetadata icon indicating copy to clipboard operation
OpenMetadata copied to clipboard

Ingestion dbt : handle multiple GCP projects / database while ingesting manifest

Open aballiet opened this issue 10 months ago • 0 comments

Is your feature request related to a problem? Please describe.

Currently lineage is incomplete if dbt use source tables located in an other GCP project / database.

When using several GCP projects as sources, meaning several serviceName, we end up with error : MY_OTHER_GCP_PROJECT.DATASET.TABLE' not found in OpenMetadata Please check if the table exists and is ingested in OpenMetadataAlso name, database, schema of the manifest node matches with the table present in OpenMetadata whereas I have this table ingested in OMD.

Describe the solution you'd like

Ingestion should map the GCP_PROJECT_ID or database_name while creating the lineage instead of solely looking among entities from serviceName.

Describe alternatives you've considered

No real alternative

Additional context Add any other context or screenshots about the feature request here.

aballiet avatar Apr 19 '24 14:04 aballiet