amundsen
amundsen copied to clipboard
Add TableLastUpdatedExtractor for BigQuery
Currently we have HiveTableLastUpdatedExtractor that figures out when the table is last modified for Hive. This issue is to support the same feature to BigQuery.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Hi! On the docs we can see that for TableLastUpdated
:
you will need access to history that provides information on when the last data write happened on a given table.
My guess is the implementation will have to leverage LAST_MODIFIED_TIME
from the INFORMATION_SCHEMA
? Any thoughts on how to implement this?
cc. @feng-tao @jinhyukchang @Golodhros
Hi,
I have done this within the BigQueryMetadataExtractor, since the api call in which BigQueryMetadataExtractor making, provides the Last Modified details.
last_modified_time=table.get('lastModifiedTime', '')
However, I am not sure how to bring this to the UI. Can anyone give some direction for that?
@feng-tao @jinhyukchang @Golodhros @dschaller @manesioz