schema_changes_from_baseline test doesn't work for Databricks

[Almaz-Murzabekov]

Describe the bug An elementary's schema_changes_from_baseline doesn't work for the Databricks delta tables. See the steps to reproduce

The test definition is in yml file (under the dbt project). Note WRONG_TYPE for the data_type parameter was set to demo the test bug (it doesn't work for any other types such as date, timestamp etc

      - name: src_appsflyer
        external:
          location: 's3://$path'
          using: delta
        tests:
          - elementary.schema_changes_from_baseline:
              fail_on_added: true
              enforce_types: true
        columns:
          - name: advertising_id
            data_type: WRONG_TYPE

The table structure from databricks metastore

how I run the dbt test command

dbt test -s source:pa.src_appsflyer --target prod

What’s the command output

07:29:47  Running 1 on-run-start hook
07:29:48  1 of 1 START hook: elementary.on-run-start.0 ................................... [RUN]
07:29:48  1 of 1 OK hook: elementary.on-run-start.0 ...................................... [OK in 0.00s]
07:29:48  
07:29:49  Concurrency: 1 threads (target='prod')
07:29:49  
07:29:49  1 of 2 START test elementary_source_schema_changes_from_baseline_pa_src_appsflyer_True__True  [RUN]
07:29:56  1 of 2 PASS elementary_source_schema_changes_from_baseline_pa_src_appsflyer_True__True  [PASS in 7.43s]
07:29:56  2 of 2 START test src_appsflyer__recency ....................................... [RUN]
07:29:59  2 of 2 PASS src_appsflyer__recency ............................................. [PASS in 3.35s]
07:29:59  
07:29:59  Running 1 on-run-end hook
07:30:12  1 of 1 START hook: elementary.on-run-end.0 ..................................... [RUN]
07:30:12  1 of 1 OK hook: elementary.on-run-end.0 ........................................ [OK in 0.00s]
07:30:12  
07:30:12  
07:30:12  Finished running 2 tests, 2 hooks in 0 hours 0 minutes and 38.50 seconds (38.50s).
07:30:12  
07:30:12  Completed successfully

How I generated baseline column definition

dbt run-operation elementary.generate_schema_baseline_test --args ‘{“name”:“src_appsflyer”, “fail_on_added”: true, “enforce_types”: true}’ -t prod

Expected behavior

dbt test will fail

Screenshots If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

Elementary version 0.10.0
dbt-core: 1.5.1
dbt-spark: 1.5.0
dbt-databricks: 1.5.4

Additional context Add any other context about the problem here.