data-engineering-zoomcamp icon indicating copy to clipboard operation
data-engineering-zoomcamp copied to clipboard

Published 20 hours ago • verified publisher icon DataTalksClub

Timecodes for "DE Zoomcamp 3.3.3 - Integrating BigQuery with Airflow (+ Week 2 Review)"

Open alexeygrigorev opened this issue 2 years ago • 1 comments

Youtube video: https://www.youtube.com/watch?v=lAxAhHNeGww

alexeygrigorev avatar May 20 '22 05:05 alexeygrigorev

0:00:02 - Automating steps, extending airflow pipeline 0:03:15 - Unnecessary services, single node setup 0:06:33 - Developing concepts, data injection, parallelization 0:10:11 - Custom setup with multiple containers 0:13:45 - Import libraries, variables, and constants 0:17:58 - Reorganize files using GCS 0:21:38 - GCS task and file movement 0:26:08 - Sorting data, creating external table 0:30:08 - Create partitioned table based on external table 0:34:08 - External data errors, directory fix 0:39:15 - Schema error in external data 0:43:00 - External data configuration and operators 0:47:56 - Workflow steps, file reorganization, query performance

dimzachar avatar Sep 09 '23 17:09 dimzachar

Updated! Thank you!

amitfrancis avatar Jan 15 '24 11:01 amitfrancis