ethereum-etl-airflow icon indicating copy to clipboard operation
ethereum-etl-airflow copied to clipboard

Published 20 hours ago • verified publisher icon blockchain-etl

Add DAG that queries data from BigQuery and saves it to csv

Open medvedev1088 opened this issue 3 years ago • 0 comments

The dag should run daily and query the last day's partition, save it to a temporary table and extract the result to a GCS bucket in gzipped csvs. The following tables should be exported:

  • blocks
  • transactions
  • traces
  • logs
  • token_transfers
  • contracts

The dag should have sensor to wait for checkpoint file in GCS, which is implemented here https://github.com/blockchain-etl/ethereum-etl-airflow/pull/142/files.

medvedev1088 avatar Oct 13 '20 14:10 medvedev1088