[BUG] Airbyte metadata ingestion pipeline - filtering the internal ID instead of pipeline name

[SChalupaBraiins]

Affected module Airbyte connector or/and Ingestion Framework

Describe the bug The Pipeline Filter (Include, Exclude) in Airbyte Metadata Ingestion does filter the Airbyte internal IDs, not the pipeline name (connection name in Airbyte terminology). This is useless, internal IDs cannot be defined by regexps and changes on every environments or re-deploy.

To Reproduce

Filtering by Pipeline names does not work. Filtering works by ID strings like 'd3720495-8668-432d-b5cf-cc350b94af2b', you can extract them from Airbyte connection URL

Expected behavior Using Airbyte pipeline name for filtering is necessary. The name is retrieved by Metadata Ingestion already, OpenMetadata use it as the pipeline name in UI.

Version:

• OS: Linux
• Python version:
• OpenMetadata version: 1.5.5
• OpenMetadata Ingestion package version: docker.getcollate.io/openmetadata/ingestion:1.5.5