dbt-clickhouse
dbt-clickhouse copied to clipboard
First version of distributed materialization
NOTE: lightly tested (though successfully). Need help from community.
Made two materializations: distributed_table for distributed version of table models and distributed_incremental for incremental models.
The main idea is to make local tables on cluster with engine and parameters from config in model's sql file within a distributed table "looking" on these local tables. So when referencing model with distributed materialization dbt will make insert into distributed table and select from distributed table. Sharding key is also to be set on model's config.
Main flow described in comments distributed_incremental.sql
There were some issues with schemas. Fixed it
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.
:white_check_mark: Volodin-DD
:x: Dimitry Volodin
Dimitry Volodin seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.
Closed in favor of #172 and #186