Add Input/Output ports for Data Product

[harshach]

Describe your changes:

This PR implements input and output port functionality for Data Products to support Data Mesh architecture patterns. Data products can now explicitly declare which data assets they consume (input ports) and which data assets they produce or expose (output ports).

---

Summary by Gitar

Data Model & Schema Changes:

• Added inputPorts and outputPorts fields to DataProduct entity as EntityReference lists
• Introduced new relationship types INPUT_PORT and OUTPUT_PORT in entityRelationship.json
• Updated Elasticsearch mappings across all languages (en, jp, ru, zh) to index port metadata

Backend Implementation:

• Implemented relationship-based storage in DataProductRepository with bulk operations (add/remove input/output ports)
• Added REST API endpoints: PUT /dataProducts/{name}/inputPorts/{add|remove} and PUT /dataProducts/{name}/outputPorts/{add|remove}
• Integrated port data into search indexes with proper field exclusions and change event generation

SDK & Client Support:

• Java SDK: Added fluent API methods in DataProducts class for port management
• Python SDK: Implemented port operations in domain_mixin and dataproducts modules

Testing Coverage:

• 15+ new test methods covering CRUD operations, bulk operations, field-based retrieval, and search propagation
• Tests validate multiple entity types as ports (tables, topics, dashboards, pipelines)

---

Type of change:

• [ ] Bug fix
• [ ] Improvement
• [x] New feature
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [ ] Documentation

Checklist:

• [x] I have read the CONTRIBUTING document.
• [ ] My PR title is Fixes <issue-number>: <short explanation>
• [ ] I have commented on my code, particularly in hard-to-understand areas.
• [ ] For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

• [ ] The issue properly describes why the new feature is needed, what's the goal, and how we are building it. Any discussion or decision-making process is reflected in the issue.
• [ ] I have updated the documentation.
• [x] I have added tests around the new logic.

[github-actions[bot]]

TypeScript types have been updated based on the JSON schema changes in the PR

[gitar-bot[bot]]

CI failed for 478fac5: Both Trivy build-and-scan jobs failed due to disk space exhaustion on CI runners - this is an infrastructure issue unrelated to the PR changes.

Issue

Both build-and-scan CI jobs (56415225010 and 56415225106) failed during Trivy security scanning with identical errors.

Root Cause

Disk space exhaustion on the GitHub Actions runner:

FATAL Fatal error run error: image scan error: scan error: scan failed: 
failed analysis: analyze error: pipeline error: failed to analyze layer: 
unable to get uncompressed layer: failed to get the layer: unable to populate: 
unable to open: failed to copy the image: 
write /tmp/trivy-14953/docker-export-3682722584: no space left on device

Details

• Job 56415225010: Failed writing to /tmp/trivy-14953/docker-export-3682722584
• Job 56415225106: Failed writing to /tmp/trivy-14311/docker-export-3872610031
• Both failures occurred during Docker image layer analysis by Trivy
• The CI runner's /tmp directory ran out of available disk space while extracting and scanning container image layers

Relationship to PR Changes

This failure is NOT related to the PR changes. This PR modifies:

• Data Product backend functionality (Java services)
• Python SDK for data products
• JSON schemas and TypeScript type definitions
• Elasticsearch index mappings

None of these changes affect Docker image building, Trivy scanning behavior, or CI runner disk usage.

Suggested Fix

The solution is to retry the CI jobs, as this is a transient infrastructure issue. The CI runner may have been in a degraded state or had accumulated temporary files from previous builds. A fresh runner should have sufficient disk space.

Reply with Gitar please fix to apply these suggestions.
_{Auto-apply is off - enable it here to let Gitar commit future updates automatically.
Did we get this right? 👍 / 👎 to help improve future analysis.}

[github-actions[bot]]

Jest test Coverage

UI tests summary

Lines	Statements	Branches	Functions
	64.3% (50881/79129)	41.9% (24774/59130)	45.39% (7806/17199)

[github-actions[bot]]

TypeScript types have been updated based on the JSON schema changes in the PR

[github-actions[bot]]

🛡️ TRIVY SCAN RESULT 🛡️

Target: openmetadata-ingestion-base-slim:trivy (debian 12.12)

Vulnerabilities (6)

Package	Vulnerability ID	Severity	Installed Version	Fixed Version
libpng-dev	CVE-2025-64720	🚨 HIGH	1.6.39-2	1.6.39-2+deb12u1
libpng-dev	CVE-2025-65018	🚨 HIGH	1.6.39-2	1.6.39-2+deb12u1
libpng-dev	CVE-2025-66293	🚨 HIGH	1.6.39-2	1.6.39-2+deb12u1
libpng16-16	CVE-2025-64720	🚨 HIGH	1.6.39-2	1.6.39-2+deb12u1
libpng16-16	CVE-2025-65018	🚨 HIGH	1.6.39-2	1.6.39-2+deb12u1
libpng16-16	CVE-2025-66293	🚨 HIGH	1.6.39-2	1.6.39-2+deb12u1

🛡️ TRIVY SCAN RESULT 🛡️

Target: Java

Vulnerabilities (33)

Package	Vulnerability ID	Severity	Installed Version	Fixed Version
com.fasterxml.jackson.core:jackson-core	CVE-2025-52999	🚨 HIGH	2.12.7	2.15.0
com.fasterxml.jackson.core:jackson-core	CVE-2025-52999	🚨 HIGH	2.13.4	2.15.0
com.fasterxml.jackson.core:jackson-databind	CVE-2022-42003	🚨 HIGH	2.12.7	2.12.7.1, 2.13.4.2
com.fasterxml.jackson.core:jackson-databind	CVE-2022-42004	🚨 HIGH	2.12.7	2.12.7.1, 2.13.4
com.google.code.gson:gson	CVE-2022-25647	🚨 HIGH	2.2.4	2.8.9
com.google.protobuf:protobuf-java	CVE-2021-22569	🚨 HIGH	3.3.0	3.16.1, 3.18.2, 3.19.2
com.google.protobuf:protobuf-java	CVE-2022-3509	🚨 HIGH	3.3.0	3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java	CVE-2022-3510	🚨 HIGH	3.3.0	3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java	CVE-2024-7254	🚨 HIGH	3.3.0	3.25.5, 4.27.5, 4.28.2
com.google.protobuf:protobuf-java	CVE-2021-22569	🚨 HIGH	3.7.1	3.16.1, 3.18.2, 3.19.2
com.google.protobuf:protobuf-java	CVE-2022-3509	🚨 HIGH	3.7.1	3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java	CVE-2022-3510	🚨 HIGH	3.7.1	3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java	CVE-2024-7254	🚨 HIGH	3.7.1	3.25.5, 4.27.5, 4.28.2
com.nimbusds:nimbus-jose-jwt	CVE-2023-52428	🚨 HIGH	9.8.1	9.37.2
com.squareup.okhttp3:okhttp	CVE-2021-0341	🚨 HIGH	3.12.12	4.9.2
commons-beanutils:commons-beanutils	CVE-2025-48734	🚨 HIGH	1.9.4	1.11.0
commons-io:commons-io	CVE-2024-47554	🚨 HIGH	2.8.0	2.14.0
dnsjava:dnsjava	CVE-2024-25638	🚨 HIGH	2.1.7	3.6.0
io.netty:netty-codec-http2	CVE-2025-55163	🚨 HIGH	4.1.96.Final	4.2.4.Final, 4.1.124.Final
io.netty:netty-codec-http2	GHSA-xpw8-rcwv-8f8p	🚨 HIGH	4.1.96.Final	4.1.100.Final
io.netty:netty-handler	CVE-2025-24970	🚨 HIGH	4.1.96.Final	4.1.118.Final
net.minidev:json-smart	CVE-2021-31684	🚨 HIGH	1.3.2	1.3.3, 2.4.4
net.minidev:json-smart	CVE-2023-1370	🚨 HIGH	1.3.2	2.4.9
org.apache.avro:avro	CVE-2024-47561	🔥 CRITICAL	1.7.7	1.11.4
org.apache.avro:avro	CVE-2023-39410	🚨 HIGH	1.7.7	1.11.3
org.apache.derby:derby	CVE-2022-46337	🔥 CRITICAL	10.14.2.0	10.14.3, 10.15.2.1, 10.16.1.2, 10.17.1.0
org.apache.ivy:ivy	CVE-2022-46751	🚨 HIGH	2.5.1	2.5.2
org.apache.mesos:mesos	CVE-2018-1330	🚨 HIGH	1.4.3	1.6.0
org.apache.thrift:libthrift	CVE-2019-0205	🚨 HIGH	0.12.0	0.13.0
org.apache.thrift:libthrift	CVE-2020-13949	🚨 HIGH	0.12.0	0.14.0
org.apache.zookeeper:zookeeper	CVE-2023-44981	🔥 CRITICAL	3.6.3	3.7.2, 3.8.3, 3.9.1
org.eclipse.jetty:jetty-server	CVE-2024-13009	🚨 HIGH	9.4.56.v20240826	9.4.57.v20241219
org.lz4:lz4-java	CVE-2025-12183	🚨 HIGH	1.8.0	1.8.1

🛡️ TRIVY SCAN RESULT 🛡️

Target: Node.js

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: Python

Vulnerabilities (3)

Package	Vulnerability ID	Severity	Installed Version	Fixed Version
starlette	CVE-2025-62727	🚨 HIGH	0.48.0	0.49.1
urllib3	CVE-2025-66418	🚨 HIGH	1.26.20	2.6.0
urllib3	CVE-2025-66471	🚨 HIGH	1.26.20	2.6.0

🛡️ TRIVY SCAN RESULT 🛡️

Target: /etc/ssl/private/ssl-cert-snakeoil.key

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/extended_sample_data.yaml

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/lineage.yaml

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/sample_data.json

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/sample_data.yaml

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/sample_data_aut.yaml

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/sample_usage.json

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/sample_usage.yaml

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/sample_usage_aut.yaml

No Vulnerabilities Found

[github-actions[bot]]

🛡️ TRIVY SCAN RESULT 🛡️

Target: openmetadata-ingestion:trivy (debian 12.12)

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: Java

Vulnerabilities (33)

Package	Vulnerability ID	Severity	Installed Version	Fixed Version
com.fasterxml.jackson.core:jackson-core	CVE-2025-52999	🚨 HIGH	2.12.7	2.15.0
com.fasterxml.jackson.core:jackson-core	CVE-2025-52999	🚨 HIGH	2.13.4	2.15.0
com.fasterxml.jackson.core:jackson-databind	CVE-2022-42003	🚨 HIGH	2.12.7	2.12.7.1, 2.13.4.2
com.fasterxml.jackson.core:jackson-databind	CVE-2022-42004	🚨 HIGH	2.12.7	2.12.7.1, 2.13.4
com.google.code.gson:gson	CVE-2022-25647	🚨 HIGH	2.2.4	2.8.9
com.google.protobuf:protobuf-java	CVE-2021-22569	🚨 HIGH	3.3.0	3.16.1, 3.18.2, 3.19.2
com.google.protobuf:protobuf-java	CVE-2022-3509	🚨 HIGH	3.3.0	3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java	CVE-2022-3510	🚨 HIGH	3.3.0	3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java	CVE-2024-7254	🚨 HIGH	3.3.0	3.25.5, 4.27.5, 4.28.2
com.google.protobuf:protobuf-java	CVE-2021-22569	🚨 HIGH	3.7.1	3.16.1, 3.18.2, 3.19.2
com.google.protobuf:protobuf-java	CVE-2022-3509	🚨 HIGH	3.7.1	3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java	CVE-2022-3510	🚨 HIGH	3.7.1	3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java	CVE-2024-7254	🚨 HIGH	3.7.1	3.25.5, 4.27.5, 4.28.2
com.nimbusds:nimbus-jose-jwt	CVE-2023-52428	🚨 HIGH	9.8.1	9.37.2
com.squareup.okhttp3:okhttp	CVE-2021-0341	🚨 HIGH	3.12.12	4.9.2
commons-beanutils:commons-beanutils	CVE-2025-48734	🚨 HIGH	1.9.4	1.11.0
commons-io:commons-io	CVE-2024-47554	🚨 HIGH	2.8.0	2.14.0
dnsjava:dnsjava	CVE-2024-25638	🚨 HIGH	2.1.7	3.6.0
io.netty:netty-codec-http2	CVE-2025-55163	🚨 HIGH	4.1.96.Final	4.2.4.Final, 4.1.124.Final
io.netty:netty-codec-http2	GHSA-xpw8-rcwv-8f8p	🚨 HIGH	4.1.96.Final	4.1.100.Final
io.netty:netty-handler	CVE-2025-24970	🚨 HIGH	4.1.96.Final	4.1.118.Final
net.minidev:json-smart	CVE-2021-31684	🚨 HIGH	1.3.2	1.3.3, 2.4.4
net.minidev:json-smart	CVE-2023-1370	🚨 HIGH	1.3.2	2.4.9
org.apache.avro:avro	CVE-2024-47561	🔥 CRITICAL	1.7.7	1.11.4
org.apache.avro:avro	CVE-2023-39410	🚨 HIGH	1.7.7	1.11.3
org.apache.derby:derby	CVE-2022-46337	🔥 CRITICAL	10.14.2.0	10.14.3, 10.15.2.1, 10.16.1.2, 10.17.1.0
org.apache.ivy:ivy	CVE-2022-46751	🚨 HIGH	2.5.1	2.5.2
org.apache.mesos:mesos	CVE-2018-1330	🚨 HIGH	1.4.3	1.6.0
org.apache.thrift:libthrift	CVE-2019-0205	🚨 HIGH	0.12.0	0.13.0
org.apache.thrift:libthrift	CVE-2020-13949	🚨 HIGH	0.12.0	0.14.0
org.apache.zookeeper:zookeeper	CVE-2023-44981	🔥 CRITICAL	3.6.3	3.7.2, 3.8.3, 3.9.1
org.eclipse.jetty:jetty-server	CVE-2024-13009	🚨 HIGH	9.4.56.v20240826	9.4.57.v20241219
org.lz4:lz4-java	CVE-2025-12183	🚨 HIGH	1.8.0	1.8.1

🛡️ TRIVY SCAN RESULT 🛡️

Target: Node.js

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: Python

Vulnerabilities (6)

Package	Vulnerability ID	Severity	Installed Version	Fixed Version
Werkzeug	CVE-2024-34069	🚨 HIGH	2.2.3	3.0.3
deepdiff	CVE-2025-58367	🔥 CRITICAL	7.0.1	8.6.1
ray	CVE-2025-62593	🔥 CRITICAL	2.47.1	2.52.0
starlette	CVE-2025-62727	🚨 HIGH	0.48.0	0.49.1
urllib3	CVE-2025-66418	🚨 HIGH	1.26.20	2.6.0
urllib3	CVE-2025-66471	🚨 HIGH	1.26.20	2.6.0

🛡️ TRIVY SCAN RESULT 🛡️

Target: /etc/ssl/private/ssl-cert-snakeoil.key

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /home/airflow/openmetadata-airflow-apis/openmetadata_managed_apis.egg-info/PKG-INFO

No Vulnerabilities Found

[sonarqubecloud[bot]]

Quality Gate passed for 'open-metadata-ui'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud