etl topic

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

airbytehq

Example project for consuming AWS Kinesis streamming and save data on Amazon Redshift using Apache Spark

vsouza

Apache AVRO for go

khezen

ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

Cinchoo

PHP ETL (Extract / Transform / Load) library with SOLID principles + almost no dependency.

bpolaszek

CubETL - Framework and tool for data ETL (Extract, Transform and Load) in Python (PERSONAL PROJECT / SELDOM MAINTAINED)

jjmontesl

Python based Wikidata framework for easy dataframe extraction

andrewtavis

A visual process builder

ajthinking

Actively curated list of awesome BI tools. PRs welcome!

thenaturalist

Provides guidance for fast ETL jobs, an IDataReader implementation for SqlBulkCopy (or the MySql or Oracle equivalents) that wraps an IEnumerable, and libraries for mapping entites to table columns.

jbogard