dbt-clickhouse
dbt-clickhouse copied to clipboard
Incremental update errors with `NUMBER_OF_COLUMNS_DOESNT_MATCH` with tuple unique_key
Hi, I can't create incremental table with tuple unique_key. I prepared two reproducible examples:
{{ config(order_by='(id1, id2)', engine='MergeTree()', materialized='incremental', unique_key='(id1, id2)') }}
select 1 as id1, 2 as id2
{{ config(order_by='(id1, id2)', engine='MergeTree()', materialized='incremental', unique_key='id1, id2') }}
select 1 as id1, 2 as id2
The error I get is
Code: 20. DB::Exception: Number of columns in section IN doesn't match. 2 at left, 1 at right. (NUMBER_OF_COLUMNS_DOESNT_MATCH) (version 22.1.3.7 (official build))
The reason seems to be excess brackets in generated query here:
where (id1, id2) not in (
select (id1, id2) -- '(' and ')' are breaking query here
from example_without_brackets__dbt_tmp
)
Also, after error tables *__dbt_old
, *__dbt_tmp
don't clean up from DB and next time I'm trying to --run I get error
Cache inconsistency detected: in rename, new key _ReferenceKey(database=None, schema='default', identifier='example_with_brackets__dbt_old') already in cache:
dbt.log
clickhouse-version: 22.1.3.7
dbt --version
Core:
- installed: 1.1.1
- latest: 1.1.1 - Up to date!
Plugins:
- clickhouse: 1.1.7 - Up to date!
Current workaround -- instead of using tuple set unique_key as single expression, i. e. concat(toString(id1), '_', toString(id2))