sqlmesh icon indicating copy to clipboard operation
sqlmesh copied to clipboard

Don't create temp tables at plan time unless they're actually needed

Open georgesittas opened this issue 10 months ago • 1 comments

We currently create temporary tables at plan time preemptively, even though they may not necessarily be needed. This behavior was introduced before we started to create tables on demand (i.e., if they're missing), but we can move away from it now.

For instance, the initial plan application for the example project produces the following tables in duckdb:

CREATE TABLE sqlmesh__sqlmesh_example.sqlmesh_example__full_model__3031530868(item_id INTEGER, num_orders BIGINT);
CREATE TABLE sqlmesh__sqlmesh_example.sqlmesh_example__full_model__3031530868__temp(item_id INTEGER, num_orders BIGINT);
CREATE TABLE sqlmesh__sqlmesh_example.sqlmesh_example__incremental_model__141311533(id INTEGER, item_id INTEGER, event_date DATE);
CREATE TABLE sqlmesh__sqlmesh_example.sqlmesh_example__incremental_model__141311533__temp(id INTEGER, item_id INTEGER, event_date DATE);
CREATE TABLE sqlmesh__sqlmesh_example.sqlmesh_example__seed_model__1248844913(id INTEGER, item_id INTEGER, event_date DATE);
CREATE TABLE sqlmesh__sqlmesh_example.sqlmesh_example__seed_model__1248844913__temp(id INTEGER, item_id INTEGER, event_date DATE);

georgesittas avatar Apr 23 '24 18:04 georgesittas