dbt_constraints icon indicating copy to clipboard operation
dbt_constraints copied to clipboard

Performance issues while creating constraints with v1.x.x

Open erdinculuturk-nw opened this issue 5 months ago • 0 comments

Upgrading from v0.6.3 to any version >= v1.0.0 ended up with performance issues in constraint creation.

We're on dbt Cloud (versionless). During my tests, I'd built same sample model using same source data. Table creation time was always same, but there was a huge difference in time required to create constraints.

In the logs below, you can see that it took 3 mins to create a not null constraint on v1.0.0, whereas it used to be 1-2 seconds on v0.6.3.

output with v1.0.0 (same with all v1 releases)

....
....
09:56:23 Installing Snowflake-Labs/dbt_constraints
09:56:23 Installed from version 1.0.0
09:56:23 Updated version available: 1.0.3
....
....
....
09:58:11 Running dbt Constraints
09:58:12 Creating not null constraint for: <colum_name> in <model_name>
10:01:18 Creating unique key: <unique_key_name>
10:01:50 Finished dbt Constraints
10:01:50 5 of 5 START hook: dbt_constraints.on-run-end.0 ................................ [RUN]
10:01:50 5 of 5 OK hook: dbt_constraints.on-run-end.0 ................................... [OK in 0.00s]
....
....

output with v0.6.3

....
....
12:00:52 Installing Snowflake-Labs/dbt_constraints
12:00:52 Installed from version 0.6.3
12:00:52 Updated version available: 1.0.3
....
....
....
12:02:17 Running dbt Constraints
12:02:18 Creating not null constraint for: <colum_name> in <model_name>
12:02:19 Creating unique key: <unique_key_name>
12:02:21 Finished dbt Constraints
12:02:21 5 of 5 START hook: dbt_constraints.on-run-end.0 ................................ [RUN]
12:02:21 5 of 5 OK hook: dbt_constraints.on-run-end.0 ................................... [OK in 0.00s]
....

erdinculuturk-nw avatar Sep 18 '24 12:09 erdinculuturk-nw