dbt-databricks icon indicating copy to clipboard operation
dbt-databricks copied to clipboard

Persist docs fails if Delta Column Mapping is enabled and column name contains spaces

Open bilalaslamseattle opened this issue 3 years ago • 2 comments

Describe the bug

Persist docs fails if Delta Column Mapping is enabled and column name contains spaces

Steps To Reproduce

  1. Create a model with Delta Column Mapping and persist docs enabled:
{{ config(
    tblproperties={'delta.columnMapping.mode':'name'},
    persist_docs={"relation": true, "columns": true}
)}}
... <truncated> ...
  1. In the model, add a space to a column name e.g.
    select
        ...
        customers.first_name as `first name`,
        ...
  1. Run the model, and you will see the ALTER TABLE fail because it's not escaping the column name correctly:
07:52:19    
07:52:19    Renaming column is not supported in Hive-style ALTER COLUMN, please run RENAME COLUMN instead.(line 4, pos 8)
07:52:19    
07:52:19    == SQL ==
07:52:19    /* {"app": "dbt", "dbt_version": "1.1.1", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.customers"} */
07:52:19    
07:52:19        
07:52:19            alter table bilal.customers change column
07:52:19    --------^^^
07:52:19                first name
07:52:19                comment 'Customer\'s first name. PII.'
07:52:19    
07:52:19  
07:52:19  Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1

Expected behavior

I can use column names with spaces

Screenshots and log output

If applicable, add screenshots or log output to help explain your problem.

System information

The output of dbt --version:

<output goes here>

The operating system you're using:

The output of python --version:

Additional context

Add any other context about the problem here.

bilalaslamseattle avatar Jul 29 '22 07:07 bilalaslamseattle

Standard tests on this column like not null are also broken

bilalaslamseattle avatar Jul 29 '22 10:07 bilalaslamseattle

You need to define the schema with quote: true to use special characters in column names:

  • schema.yml
version: 2

models:
  - name: customers
    columns:
      - name: 'first name'
        quote: true
        description: ...
...

ueshin avatar Jul 29 '22 20:07 ueshin

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue.

github-actions[bot] avatar Jan 26 '23 01:01 github-actions[bot]