etl Type/date handler should keep track of which date partitions are updated.

Type/date handler should keep track of which date partitions are updated.

Open gfr10598 opened this issue 5 years ago • 0 comments

Part of m-lab/dev-tracker#430 KR: Simplify Gardener, ETL Tuning, and New Parser Development

Also see #732

As parser handles each tar file, it should report back to Gardener the partition dates that are updated.

Alternatively, Gardener may be able to discover this by queries to BigQuery if the overhead is low enough.

Oct 22 '19 23:10 gfr10598