dbt-clickhouse icon indicating copy to clipboard operation
dbt-clickhouse copied to clipboard

Microbatch fails on event_time columns of type Date

Open nastynaz opened this issue 3 months ago • 0 comments

Describe the bug

The current macro for microbatch expands to format the event_time as a DateTime. This causes SQL compilation errors if the column is stored as a Date in clickhouse. For microbatches with granularity at day or above it should be allowed to also work with Date columns.

The offending lines of code are here:

      {%- if config.get("__dbt_internal_microbatch_event_time_start") -%}
        {% do incremental_predicates.append(config.get("event_time") ~ " >= toDateTime('" ~ config.get("__dbt_internal_microbatch_event_time_start").strftime("%Y-%m-%d %H:%M:%S") ~ "')") %}
      {%- endif -%}
      {%- if model.config.__dbt_internal_microbatch_event_time_end -%}
        {% do incremental_predicates.append(config.get("event_time") ~ " < toDateTime('" ~ config.get("__dbt_internal_microbatch_event_time_end").strftime("%Y-%m-%d %H:%M:%S") ~ "')") %}
      {%- endif -%}

Notice the coercion via strftime("%Y-%m-%d %H:%M:%S")

Steps to reproduce

  1. Create a table with a Date column.
  2. Create a dbt model of the table and set event_time to the column which is Date.
  3. Try and run it. It will fail

Expected behaviour

event_time should work with Date columns.

nastynaz avatar Sep 25 '25 19:09 nastynaz