OpenMetadata icon indicating copy to clipboard operation
OpenMetadata copied to clipboard

Add OCI image repositories as

Open suse-coder opened this issue 5 months ago • 0 comments

Is your feature request related to a problem? Please describe.
Many Data source nowadays for ML and data science (KitOps) come from oci images (harbor, artifactory, gitlab). It would be great if openmetadata would also support to add that as connector.

Describe the solution you'd like
I would like OpenMetadata to support OCI (Open Container Initiative) image registries as data sources. This means being able to add connectors for popular OCI-compatible registries such as Harbor, Artifactory, and GitLab Container Registry. The connector should allow users to discover, catalog, and manage metadata for datasets and artifacts stored within OCI images, similar to how OpenMetadata integrates with databases, data lakes, and other sources.

Describe alternatives you've considered

  • Using custom scripts or manual processes to extract metadata from OCI images and then ingesting it into OpenMetadata via the API.
  • Creating an external metadata catalog specifically for OCI images and integrating that catalog with OpenMetadata, which adds complexity and maintenance overhead.
  • Relying on existing connectors for file storage (e.g., S3, GCS) but these do not natively understand OCI image structure or metadata.

Additional context

  • OCI images are increasingly used for packaging data and ML models, especially in MLOps and DataOps workflows.
  • Native support in OpenMetadata would streamline governance, discovery, and lineage tracking for ML/data science assets.
  • Example: A data science team stores trained models and datasets as OCI images in Harbor. With a native connector, OpenMetadata could automatically extract and catalog metadata such as image tags, creation date, authors, and embedded dataset/model details.
  • Screenshots or diagrams can be provided if needed to illustrate the desired integration flow.

suse-coder avatar Jun 14 '25 09:06 suse-coder