dbt-checkpoint icon indicating copy to clipboard operation
dbt-checkpoint copied to clipboard

precommit fails on different model versions

Open jan-benisek opened this issue 2 years ago • 4 comments

Describe the bug

  • dbt >1.5 introduced model versions
  • the precommit hooks Check the model has properties file and Check the model has description fail
  • probably because it expects description for each version, while there is none

To Reproduce Steps to reproduce the behavior:

  1. Create two versions of the same model fancy_model_v1.sql and fancy_model_v2.sql
  2. Adapt the yml file
- name: fancy_model
  latest_version: 1
  description: fancy model
  config:
    contract:
      enforced: true
  columns:
    - name: tool_raw
      description: raw name of tool
      data_type: varchar(20)
      constraints:
        - type: not_null
        - type: primary_key
          warn_unenforced: False
    - name: tool
      description: tool name, "jpg-to-pdf"
      data_type: varchar(20)

  versions:
    - v: 1
    - v: 2
      columns:
        - include: all
          exclude: [tool]
        - name: new_column
          data_type: varchar(6)
          description: "I am an experimental column"
  1. Run the Check the model has properties file and Check the model has description
  2. The error is:
#14 [dbt_precommit 4/4] RUN pre-commit run --all-files
...
#14 1.186 Check the model has properties file......................................Failed
#14 8.483 - hook id: check-model-has-properties-file
#14 8.483 - exit code: 1
#14 8.483 models/fancy_model_v1.sql: does not have model properties defined in any .yml file.
#14 8.483 models/fancy_model_v2.sql: does not have model properties defined in any .yml file.
...
#14 8.484 Check the model has description..........................................Failed
#14 34.62 - hook id: check-model-has-description
...
#14 34.62 models/fancy_model_v1.sql: does not have defined description or properties file is missing.
#14 34.62 models/fancy_model_v2.sql: does not have defined description or properties file is missing.

Expected behavior

  • precommit should understand that fancy_model_v1.sql and fancy_model_v2.sql are the same and have description under fancy_model in yml file

Version: v0.1.0

Additional context Add any other context about the problem here.

jan-benisek avatar Jul 11 '23 15:07 jan-benisek

Hi @jan-benisek , I was able to recreate the issue in a personal project and solve for the issue after running a pre-commit autoupdate, perhaps your version of pre-commit is outdated? Curious to see if that solves your issue.

alfredodimassimo avatar Aug 24 '23 20:08 alfredodimassimo

Thanks for the response @alfredodimassimo, I updated to pre-commit=3.3.3 and pre-commit-hooks version v4.4.0 but I am still getting the same error on the versioned models.

jan-benisek avatar Aug 25 '23 06:08 jan-benisek

Issue can be reproduced with dbt version 1.5.0. However it is "fixed" by how dbt 1.6.0 handles semantic models.

The difference is that dbt ls -s fancy_model_v2 in 1.5 returns nothing, but 1.6 returns the node fancy_model.v2.

diorge avatar Sep 19 '23 13:09 diorge

I'm experiencing this issue with dbt v1.7.5, pre-commit v3.6.0, and dbt-checkpoint v1.2.0.

mijns avatar Jan 23 '24 16:01 mijns

Same here: dbt 1.7.9 pre-commit 3.6.1 dbt-checkpoint 1.2.1

CommonCrisis avatar Mar 13 '24 14:03 CommonCrisis