dbt-databricks

dbt-databricks copied to clipboard

Feature/python model beta

1

### Description Ports the changes for python model v1 from [`dbt-spark`](https://github.com/dbt-labs/dbt-spark/pull/377) but use APIs below instead. - [Create an execution context](https://docs.databricks.com/dev-tools/api/1.2/index.html#create-an-execution-context ) - [Run a command](https://docs.databricks.com/dev-tools/api/1.2/index.html#run-a-command) - [Get information about...

ueshin

resolves #160 ### Description Updates the incremental materialization to include user supplied `incremental_predicates` and passes that to the `dbt_spark_get_incremental_sql` macro. The changes to `dbt_spark_get_incremental_sql` are updated in [dbt-spark PR #436](https://github.com/dbt-labs/dbt-spark/pull/436)...

dave-connors-3

support user incremental predicates on merge

1

### Describe the feature Users should be able to supply incremental models with arbitrary predicates during the merge step to prevent unnecessary table scan for massive data sets. ### Describe...

dave-connors-3

'append_new_columns' flag causes failures for 'append' incremental models

2

### Describe the bug When using the `append_new_columns` flag for incremental models with an `append` incremental strategy, the first `dbt run` adding new columns will fail. However, the next run...

TannerHopkins

Apply connection retry refactor, add defaults with exponential backoff

1

### Description Applies connection retry refactor, add defaults with exponential backoff as per an item from #127. - Refactors retry logic to use `retry_connection` from core. - Adds consistent defaults...

ueshin

Connection test: [ERROR] - dbt-databricks behind proxy

9

### Describe the bug A clear and concise description of what the bug is. What command did you run? What happened? dbt debug gives error Connection test: [ERROR] 1 check...

thuanvan

Support for partition overwrite with Delta

7

### Describe the feature _This was reported in https://github.com/dbt-labs/dbt-spark/issues/155 but I think you might be more interested in resolving the issue_ > Currently, the insert_overwrite strategy throws an error if...

dejan

Add support for identity columns

6

### Describe the feature Now that [identity columns ](https://www.databricks.com/blog/2022/08/08/identity-columns-to-generate-surrogate-keys-are-now-available-in-a-lakehouse-near-you.html) are GA, it would be great to be able to specify the identity column as part of the model configuration. ###...

flvndh

resolves #139 ### Description Make `table` and `incremental` with `full-refresh` check the external location.

ueshin

Persist docs fails if Delta Column Mapping is enabled and column name contains spaces

2

### Describe the bug Persist docs fails if Delta Column Mapping is enabled and column name contains spaces ### Steps To Reproduce 1. Create a model with [Delta Column Mapping](https://docs.databricks.com/delta/delta-column-mapping.html)...

bilalaslamseattle