data-engineering-pipeline topic

List data-engineering-pipeline repositories

xontrib-pipeliner

56
Stars
5
Forks
Watchers

Let your pipe lines flow thru the Python code in xonsh.

nyc-stats

20
Stars
1
Forks
Watchers

Analysis of 311 Service Requests for the City of NYC (from 2010 to 2023) Tech: Prefect cloud, dbt core, BigQuery, Compute Engine, CloudRun, Artifact Registry, Terraform, Docker

data-engineer-roadmap

15
Stars
10
Forks
Watchers

Learning from multiple companies in Silicon Valley. Netflix, Facebook, Google, Startups

Gear5

16
Stars
4
Forks
Watchers

high performance better alternative to Airbyte, Singer, Meltano

A end-to-end real-time stock market data pipeline with Python, AWS EC2, Apache Kafka, and Cassandra Data is processed on AWS EC2 with Apache Kafka and stored in a local Cassandra database.