Support disabling unit tests

[tsturge]

Support config: disabled for unit tests. (#9109)

Also, when a model is disabled, disable the corresponding unit tests automatically. (#10540)

Problem

config: enabled: false

Solution

Check for the config: enabled variable while parsing the unit test. If it is set to false, mark the unit test as disabled.

Checklist

• [x] I have read the contributing guide and understand what's expected of me.
• [x] I have run this code in development, and it appears to resolve the stated issue.
• [x] This PR includes tests, or tests are not required or relevant for this PR.
• [x] This PR has no interface changes (e.g., macros, CLI, logs, JSON artifacts, config files, adapter interface, etc.) or this PR has already received feedback and approval from Product or DX.
• [x] This PR includes type annotations for new and modified functions.

[cla-bot[bot]]

Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA.

In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR.

CLA has not been signed by users: @tsturge

[github-actions[bot]]

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 71.42857% with 4 lines in your changes missing coverage. Please review.

Project coverage is 89.13%. Comparing base (dd77210) to head (56f2300). Report is 59 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #10831      +/-   ##
==========================================
- Coverage   89.19%   89.13%   -0.07%     
==========================================
  Files         183      183              
  Lines       23496    23508      +12     
==========================================
- Hits        20958    20953       -5     
- Misses       2538     2555      +17     

Flag	Coverage Δ
integration	86.44% <71.42%> (-0.14%)	:arrow_down:
unit	62.11% <42.85%> (+0.02%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
Unit Tests	62.11% <42.85%> (+0.02%)	:arrow_up:
Integration Tests	86.44% <71.42%> (-0.14%)	:arrow_down:

[cla-bot[bot]]

Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA.

In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR.

CLA has not been signed by users: @tsturge

[devmessias]

I like this. Especially because the unit tests are now broken when we have ctx like vars inside the model. The way I was handling this was by commenting them out.

[maxmarcon]

Hi! I was eagerly waiting for this to be released. I saw that this PR was merged 3 weeks ago. A new dbt-core version (1.8.9) was released one week ago but this feature doesn't seem to work there yet. This unit test:

version: 2

unit_tests:
  - name: consolidate_gmv_currencies
    config: 
      enabled: false 
    model: validate_consolidate_gmv_currencies

gets still listed by dbt list

Am I missing something? Thanks

[davidbooke4]

Hi! I was eagerly waiting for this to be released. I saw that this PR was merged 3 weeks ago. A new dbt-core version (1.8.9) was released one week ago but this feature doesn't seem to work there yet. This unit test:

version: 2

unit_tests:
  - name: consolidate_gmv_currencies
    config: 
      enabled: false 
    model: validate_consolidate_gmv_currencies

gets still listed by dbt list

Am I missing something? Thanks

@maxmarcon the changes in this PR weren't included in 1.8.9, but they are included in 1.9.0. I've tested out disabling them as you have above, and it's working for me.

[davidbooke4]

While explicitly disabling a unit test using the below code is working:

version: 2

unit_tests:
  - name: consolidate_gmv_currencies
    config: 
      enabled: false 
    model: validate_consolidate_gmv_currencies

I'm still getting parsing errors when a model is disabled and a unit test is not explicitly disabled:

Parsing Error
  Unable to find model 'disabled_model' for unit test 'unit_test_for_disabled_model' in ...

Whether I have the model disabled using conditional logic such as config(enabled=var('model_enabled', false) != false) or config(enabled = false) doesn't make a difference.

It seems like the unit test should be disabled in this situation based on the changes made in core/dbt/parser/unit_tests.py in this PR. It looks like the unit tests added in this PR are testing for when unit tests are disabled and not when models are disabled, but I could be wrong about that.

I'm testing this out on dbt-core v1.9.0 and dbt-bigquery v1.9.0. Please let me know if I'm doing something wrong - thanks!

[srggrs]

Hi guys do you plan to backport this into v1.8 minor release, we are currently blocked on upgrading to v1.9 as our other OSS parts won't work with v1.9. Thanks 🙏🏼