dbt-datamocktool icon indicating copy to clipboard operation
dbt-datamocktool copied to clipboard

Failed to replace if input_mapping keys do not exactly match with the reference in the sql file

Open NGL91 opened this issue 1 year ago • 2 comments

Describe the bug

In older versions, the input mapping keys are rendered and then replace in the rendered SQL. However, in newer versions, the input mapping keys are replaced in the raw SQL. Therefore, the format of the input mapping keys must exactly match the format in the SQL file in order to be find and replace.

Steps to reproduce

input_mappings:
   ref('model'): ref("mock_reference')

SQL file:

SELECT * FROM {{ ref("model") }}

datamocktool can't replace ref('model') with ref("mock_reference') for the SQL file

Expected results

  • datamocktool replace ref("model") with ref("mock_reference') during the test.
  • Or raises error if any of the input mapping keys are not found in the SQL file.

Actual results

  • ref("model") is not replaced during test.
  • No warning/error raises for mismatch between input_mapping keys with the reference in the sql.

Screenshots and log output

System information

The contents of your packages.yml file:

Which database are you using dbt with?

  • [ ] postgres
  • [ ] redshift
  • [ x] bigquery
  • [ ] snowflake
  • [ ] other (specify: ____________)

The output of dbt --version:

1.5.1

Additional context

Are you interested in contributing the fix?

yes

NGL91 avatar Jul 31 '23 02:07 NGL91

We have seen something similar on BigQuery and dbt 1.5.x. Interested to figure out how this will be solved...

anderseriksson avatar Sep 21 '23 10:09 anderseriksson

I am also experiencing this.

Even worse, now in 0.3.5 It's also the case that even if the ref is a perfect match, if located within a Jinja macro it won't find it anymore. 0.3.4 seems good.

select * from
    {{
        deduplicate(
            relation=ref("mock_ref"),
            partition_by='mock_dim',
        )
    }}

@mjirv I believe the root cause is the changes in the file mentioned here

lrodriguezesc avatar Jul 11 '24 17:07 lrodriguezesc