dbt-core
dbt-core copied to clipboard
Replace DBT_EXPERIMENTAL_MICROBATCH env var with behaviour flag
Prior to the 1.9 beta, we should:
-
Replace usage of env var
DBT_EXPERIMENTAL_MICROBATCHwith a project flag,require_builtin_microbatch_strategywhich defaults toFalse(@graciegoheen approves the name :) ) -
Make changes to dbt-adapters:
- this needs to use the project flag instead of os env var: https://github.com/dbt-labs/dbt-adapters/blob/b414668cd046ce9b68c03b68ca7fc94f190b4e99/dbt/adapters/base/impl.py#L1573
- base test needs to use the new project flag instead of env var: https://github.com/dbt-labs/dbt-adapters/blob/b414668cd046ce9b68c03b68ca7fc94f190b[…]er/dbt/tests/adapter/incremental/test_incremental_microbatch.py
- example: https://github.com/dbt-labs/dbt-core/pull/10799/files#diff-ac70d9689da91b61cf649606ebbc0cd875e46dfd8c3627dd8f8d124e08f60fd5R86-R93
- Make changes to dbt-bigquery:
- a couple additional tests have been added beyond the base test the additional tests patch the os env var directly
- https://github.com/dbt-labs/dbt-bigquery/blob/main/tests/functional/adapter/incremental/test_incremental_microbatch.py#L33
- https://github.com/dbt-labs/dbt-bigquery/blob/main/tests/functional/adapter/incremental/test_incremental_microbatch.py#L48 they'll need to be updated to use the project_config_update fixture instead
dbt-postgres and dbt-snowflake use the base test only and don't need any manual updates