dbt-databricks

dbt-databricks copied to clipboard

Passing HTTP User Agent Tag to databricks by dbt

3

### Describe the feature As per Databricks best practices, Passing the UserAgent attribute is required for all integrations with Databricks. The UserAgent attribute denotes the product/vendor name. I am not...

balkrushnapatil

Performance improvement on persist_docs

10

### Describe the feature persist_docs currently sends an individual alter table statement per column comment. This takes forever for a large project. ### Describe alternatives you've considered Don't use persist_docs...

jordandakota

Show command with nested json yields "Object of type ndarray is not JSON serializable"

2

### Describe the bug dbt-databricks throws Object of type ndarray is not JSON serializable when previewing nested json (`dbt show`) ### Steps To Reproduce * Open a databricks dbt project...

racheldaniel

Character encoding changes upon seeding

1

### Describe the bug A CSV file that has UTF-8 encoding is seeded with `dbt seed`. Upon review of the load, the column encoding has appeared to change. ### Steps...

jdbodyfelt

Add support for `INSERT ... BY NAME`

4

### Describe the feature Starting from Databricks Runtime 13.3 and above, it is possible to use the `BY NAME` clause instead of an explicit column list in an `INSERT` statement...

flvndh

dbt test removes Delta Transaction Log history after every run

1

### Describe the bug I don't understand the behavior of dbt-Databricks. Delta transaction log is always refreshed after every `dbt test` run. Does dbt always rollback test results i.e. delete...

ammarchalifah-bolt

Support new `constraints` spec, and specifying constraints in `create table as`

1

(Happy to split this into multiple issues — keeping it together here for now, to keep the conversation centralized) ### Describe the feature First, we are changing the spec of...

jtcohen6

extend `copy_into` to take information from `sources`

1

### Describe the feature It would be useful to extend `copy_into` to take information from `sources`, similar to [external](https://docs.getdbt.com/reference/resource-properties/external) support - it would significantly simplify developer's life. ### Who will...

alexott

Support for IAM credential passthrough clusters

1

My team uses iam credential passthrough on our clusters to access data that sits in s3. This requires us to call `SET spark.databricks.credentials.assumed.role=` as a pre-hook to models to be...

th3ed

bigint values in seeds >= 16 digits get mangled in `dbt seed`

2

### Describe the bug A seed with field values that should be properly represented as bigints gets populated to the destination table incorrectly. This seems to happen with values exceeding...

otosky