[spike] Allow generic data tests to be documented

[clrcrl]

Describe the feature

Currently, you can document pretty much everything but a data test - if you try, it fails on a parsing error. This ticket is for generic tests. A separate ticket addresses singular tests: https://github.com/dbt-labs/dbt-core/issues/9005

Acceptance criteria

• You can add a description to a generic test
• You can use a docs block to capture your test description
• That description shows up in the docs site

Examples

version: 2

models:
  - name: orders
    columns:
      - name: order_id
        tests:
          - unique
            description: The order_id is unique for every row in the orders model
          - not_null
            description: '{{ doc("not_null_test") }}'

Concerns

• adding documentation to tests will slow down parsing (full parse), particularly for really common tests like unique and not_null
• We have special processing to speed up unique and not_null test parsing. Would that have to change?
• Ensure the solution is performant

[RobMcZag]

I would say that for me a tests: label would imply all kind of tests, including schema tests, but looking at the current naming it looks like the most reasonable. Current names related to tests:

• data tests are in the tests folder, not data-tests
• in dbt_projectwe specify a folder for test-paths, not data-test-paths

[gshank]

I know we've discussed putting schema tests in the tests directory instead of the macros directory, but I don't recall what the resolution of that was. Is that on the roadmap somewhere?

[jtcohen6]

@gshank @kwigley I'd like to use our discussion of this issue as a jumping-off point to discuss other test-related work we scoped out earlier this year, and to determine whether / which of these should go on the v1.0 list as well.

The ones that come to my mind are:

• https://github.com/fishtown-analytics/dbt/issues/3234
• https://github.com/fishtown-analytics/dbt/issues/3249
• https://github.com/fishtown-analytics/dbt/issues/3250
• https://github.com/fishtown-analytics/dbt/issues/3259
• https://github.com/fishtown-analytics/dbt/issues/3348

All of those would represent a breaking change to some degree. There's ways we can offer backwards compatibility; easier for some, harder for others.

[leahwicz]

Work:

• New parser for tests
• Work will need to be done in partial parsing
• Involve doc blocks (perf hit)

[github-actions[bot]]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.

[NicolasPA]

I think the ticket activity doesn't reflect the demand for this feature, there has been at least 6 users asking about in on Slack this past year: https://getdbt.slack.com/archives/CBSQTAPLG/p1631304882432300

It might be because the wording used in the ticket has changed: schema test → generic test data test → singular test

I think custom tests are code as much as models and macros are, so documenting them is equally important.

@jtcohen6: I think a new tests: key in .yml files is exactly how we'd want to do this. Singular tests wouldn't be too hard. Generic tests would be trickier, since the good version (IMO) allows for templated descriptions that includes the values of {{ model }}, {{ column_name }}, and potentially other arguments.

IMO, having the static description as is in written in the YAML without replacing the values would already be nice and useful, the more complex version could be kept for later.

[jeff-skoldberg-ct]

I came to +1 this post. Looking forward to support for documenting Singular Tests. Thanks!

[gnilrets]

+1 from me too.

[ChristianP89]

+1 from my side as well. It's quite important to be able to document what a test does.

[nbkrd1]

+1 from me as well.

[nbllh1]

+1 from me too.

[GeorgeHC97]

+1

[ghost]

+1

[minhnhat992]

+1

[dvanbolt]

+1

[trevor-petach]

+1

[bethanyhipple-dbtlabs]

+1

[moralescastillo]

+1

[dennisgera]

+1 from me too.

[jfreels]

+1

[ipcleary]

+1

[culpgrant]

+1 from me

[mwbong]

+1 please

[simonhoerdumbonde]

+1

[scwin813]

+1

[sedgar]

+1 please

[AnotherGuitar]

+1 please!

[kgeis]

The compiler doesn't complain when I set the description in a singular test's config block, but when the docs are rendered, it says "This test is not currently documented". 🤔

When you get to this, I'd like you to add test-paths to the default doc-paths so that I don't need to do anything weird to get the compiler to pick up the doc blocks for my tests.

[rd144]

+1

[mrkjones79]

+1 from me too. Given that tests are included in the docs, and that it's just good practice, it should be possible to document singular tests.