datahub
datahub copied to clipboard
Invalid URNs generated for Metabase Dashboards with Text or Heading Cards
Describe the bug
Dashboards with virtual (Text or Heading) Cards cannot be ingested due to invalid URNs.
To Reproduce Steps to reproduce the behavior:
- Create a dashboard in Metabase
- Add a heading and/or a text element to the dashboard
- Run datahub metadata ingestion
- Notice errors
Expected behavior
Metadata can be ingested without errors
Screenshots
{'cli_version': 'unavailable (installed in develop mode)',
'cli_entry_location': '/home/pulsar/development/4screen/datahub/metadata-ingestion/src/datahub/__init__.py',
'models_version': 'bundled',
'py_version': '3.10.14 (main, Apr 25 2024, 08:20:49) [GCC 13.2.1 20230801]',
'py_exec_path': '/home/pulsar/development/4screen/datahub/metadata-ingestion/venv/bin/python',
'os_details': 'Linux-6.8.7-zen1-1-zen-x86_64-with-glibc2.39',
'mem_info': '109.77 MB',
'peak_memory_usage': '109.77 MB',
'disk_info': {'total': '965.39 GB', 'used': '616.21 GB', 'used_initally': '616.21 GB', 'free': '300.07 GB'},
'peak_disk_usage': '616.21 GB',
'thread_count': 4,
'peak_thread_count': 4}
Source (metabase) report:
{'events_produced': 2,
'events_produced_per_sec': 3,
'entities': {'dashboard': ['urn:li:dashboard:(metabase,1)']},
'aspects': {'dashboard': {'dashboardInfo': 1, 'ownership': 1, 'status': 1}},
'warnings': {},
'failures': {},
'start_time': '2024-04-25 16:39:14.176924 (now)',
'running_time': '0.56 seconds'}
Sink (datahub-rest) report:
{'total_records_written': 1,
'records_written_per_second': 1,
'warnings': [],
'failures': [{'error': 'Unable to emit metadata to DataHub GMS: com.linkedin.metadata.entity.validation.ValidationException: Failed to validate '
'record with class com.linkedin.entity.Entity: ERROR :: '
'/value/com.linkedin.metadata.snapshot.DashboardSnapshot/aspects/0/com.linkedin.dashboard.DashboardInfo/charts/0 :: '
'"Provided urn urn:li:chart:(metabase,)" is invalid\n'
'ERROR :: /value/com.linkedin.metadata.snapshot.DashboardSnapshot/aspects/0/com.linkedin.dashboard.DashboardInfo/charts/1 :: '
'"Provided urn urn:li:chart:(metabase,)" is invalid\n',
'info': {'exceptionClass': 'com.linkedin.restli.server.RestLiServiceException',
'message': 'com.linkedin.metadata.entity.validation.ValidationException: Failed to validate record with class '
'com.linkedin.entity.Entity: ERROR :: '
'/value/com.linkedin.metadata.snapshot.DashboardSnapshot/aspects/0/com.linkedin.dashboard.DashboardInfo/charts/0 '
':: "Provided urn urn:li:chart:(metabase,)" is invalid\n'
'ERROR :: '
'/value/com.linkedin.metadata.snapshot.DashboardSnapshot/aspects/0/com.linkedin.dashboard.DashboardInfo/charts/1 '
':: "Provided urn urn:li:chart:(metabase,)" is invalid\n',
'status': 422,
'urn': 'urn:li:dashboard:(metabase,1)'}}],
'start_time': '2024-04-25 16:39:13.777554 (now)',
'current_time': '2024-04-25 16:39:14.736834 (now)',
'total_duration_in_seconds': 0.96,
'max_threads': 15,
'gms_version': 'v0.13.2',
'pending_requests': 0}
Versions
Master / 826c4bff77f93a16da39d9b6b18eb4e4b28e52c5
Additional context
Text and Heading dashboard cards are "virtual cards" and have no valid Id. When DataHub Metadata ingestion tries to construct a URN for that card it will result in invalid data: urn:li:chart:(metabase,)