sagerx icon indicating copy to clipboard operation
sagerx copied to clipboard

Intermediate models as tables

Open lprzychodzien opened this issue 1 month ago • 0 comments

Explanation

Sets intermediate models to materialize as tables (vs views).

Rationale

dbt mart models are made up of complex logic that should be captured in the intermediate models. The issues already that we ran into (issue https://github.com/coderxio/sagerx/issues/270 ) was that performing these aggregations for each query took 10+ minutes. Therefore, bringing this aggregation into a separated intermediate model to be materialized as a table is a good solution. The materialization of these intermediate tables can take awhile but will significantly speed up queries.

Additional work can be done to optimize the queries that build out intermediate models in the future.

Tests

  1. What testing did you do? dbt run --full-refresh

lprzychodzien avatar May 17 '24 14:05 lprzychodzien