dbt_hubspot_source icon indicating copy to clipboard operation
dbt_hubspot_source copied to clipboard

[Bug] Quotes missing around column names and aliases when hubspot__pass_through_all_columns is enabled

Open aryan-wisdom opened this issue 5 months ago • 5 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues

Describe the issue

The models/stg_hubspot__company.sql, models/stg_hubspot__contact.sql and models/stg_hubspot__deal.sql files are not compiling because quotes are not present around column names when hubspot__pass_through_all_columns is true.

Relevant error log or model output

....

 as company_annual_revenue 
(     
        -- just pass everything through if extra columns are present, but ensure required columns are present.
        property_hs_all-deprecated-85213efe-a820-40c1-815f-4c19fed96ad7 as hs_all-deprecated-85213efe-a820-40c1-815f-4c19fed96ad7,
    from base
)

Expected behavior

....

as company_annual_revenue (
-- just pass everything through if extra columns are present, but ensure required columns are present. "property_hs_all-deprecated-85213efe-a820-40c1-815f-4c19fed96ad7" as "hs_all-deprecated-85213efe-a820-40c1-815f-4c19fed96ad7", from base )

Possible solution

A for loop around the code:

                fivetran_utils.remove_prefix_from_columns(
                columns=adapter.get_columns_in_relation(ref('stg_hubspot__company_tmp')), 
                prefix='property_', exclude=get_macro_columns(get_company_columns()))

which applies fivetran_utils.quote_column on each column name and alias would help.

dbt Project configurations

vars: hubspot__pass_through_all_columns: true

Package versions

packages:

  • package: fivetran/fivetran_utils version: [">=0.4.0", "<0.5.0"]

  • package: dbt-labs/spark_utils version: [">=0.3.0", "<0.4.0"]

What database are you using dbt with?

redshift

How are you running this dbt package?

dbt Core™

dbt Version

1.7.7

Additional Context

No response

Are you willing to open a PR to help address this issue?

  • [ ] Yes.
  • [X] Yes, but I will need assistance.
  • [ ] No.

aryan-wisdom avatar Sep 10 '24 11:09 aryan-wisdom