data-pipelines topic

List data-pipelines repositories

udacity-data-eng-proj-1

88
Stars
58
Forks
Watchers

Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation, validation and loading of data from S3 -> Redshift -> S3

pipebird

167
Stars
7
Forks
Watchers

Pipebird is open source infrastructure for securely sharing data with customers.

ml-in-production

49
Stars
22
Forks
Watchers

The practical use-cases of how to make your Machine Learning Pipelines robust and reliable using Apache Airflow.

streams-explorer

44
Stars
4
Forks
Watchers

Explore Apache Kafka data pipelines in Kubernetes.

data_engineer_interview_challenges

61
Stars
10
Forks
Watchers

Found a data engineering challenge or participated in a selection process ? Share with us!

fsharp-data-processing-pipeline

15
Stars
1
Forks
Watchers

Provides an extensible solution for creating Data Processing Pipelines in F#.

AirflowDataPipeline

31
Stars
19
Forks
Watchers

Example of an ETL Pipeline using Airflow

stepist

27
Stars
5
Forks
Watchers

Framework for data processing

neon-workshop

19
Stars
6
Forks
Watchers

A Pachyderm deep learning tutorial for conference workshops