dbt-clickhouse icon indicating copy to clipboard operation
dbt-clickhouse copied to clipboard

Dont drop tables in incremental append runs

Open stephen-up opened this issue 4 months ago • 2 comments

Hi, when dbt runs for a incremental+append it always drops the temporary tables that are used for full refreshes. Even if its not going to use them.

In my system, I have scheduled incremental runs of dbt that run frequently updating incremental+append tables. Whenever I do a full refresh (concurrently), the scheduled run drops the tables of the full refresh causing it to crash.

This PR stops that, by making the incremental+append runs not drop tables.

I see there have been a few other issues similar to this, but this seems like an easy minimal way to fix the issue.

  • https://github.com/ClickHouse/dbt-clickhouse/issues/150
  • https://github.com/ClickHouse/dbt-clickhouse/pull/353

Thanks

stephen-up avatar Oct 09 '24 00:10 stephen-up