amundsen icon indicating copy to clipboard operation
amundsen copied to clipboard

Add TableLastUpdatedExtractor for BigQuery

Open jinhyukchang opened this issue 4 years ago • 3 comments

Currently we have HiveTableLastUpdatedExtractor that figures out when the table is last modified for Hive. This issue is to support the same feature to BigQuery.

jinhyukchang avatar Jul 18 '19 18:07 jinhyukchang

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

stale[bot] avatar Aug 16 '19 05:08 stale[bot]

Hi! On the docs we can see that for TableLastUpdated:

you will need access to history that provides information on when the last data write happened on a given table.

My guess is the implementation will have to leverage LAST_MODIFIED_TIME from the INFORMATION_SCHEMA? Any thoughts on how to implement this?

cc. @feng-tao @jinhyukchang @Golodhros

manesioz avatar Aug 11 '20 14:08 manesioz

Hi,

I have done this within the BigQueryMetadataExtractor, since the api call in which BigQueryMetadataExtractor making, provides the Last Modified details.

last_modified_time=table.get('lastModifiedTime', '')

However, I am not sure how to bring this to the UI. Can anyone give some direction for that?

@feng-tao @jinhyukchang @Golodhros @dschaller @manesioz

sagar-raythatha avatar Oct 14 '21 12:10 sagar-raythatha