prefect icon indicating copy to clipboard operation
prefect copied to clipboard

Pre-built DBT Tasks and Flow

Open robfreedy opened this issue 1 year ago • 1 comments

This PR includes implementations for 4 pre-built Prefect DBT tasks as well as a pre-built Prefect DBT Flow. These pre-built tasks/flows will run the respective DBT command as a Prefect task/flow and will create a Prefect Markdown Artifact that provides a summary of the DBT runs, including any failed, successful, and skipped runs.

Closes the following issues: https://github.com/PrefectHQ/prefect-dbt/issues/179 https://github.com/PrefectHQ/prefect-dbt/issues/180 https://github.com/PrefectHQ/prefect-dbt/issues/181 https://github.com/PrefectHQ/prefect-dbt/issues/182

Example

from prefect import flow
from prefect_dbt.cli.tasks import dbt_run_task


@flow
def dbt_flow():
    dbt_run_task(
        project_dir="/Users/test/prefect_dbt_project"
    )
    dbt_seed_task(
        project_dir="/Users/test/prefect_dbt_project"
    )
    dbt_snapshot_task(
        project_dir="/Users/test/prefect_dbt_project"
    )
    dbt_test_task(
        project_dir="/Users/test/prefect_dbt_project"
    )

if __name__ == "__main__":
    dbt_flow()

from prefect import flow
from prefect_dbt.cli.flow import dbt_flow


@flow
def dbt_flow_test():
    dbt_flow(
        project_dir="/Users/test/prefect_dbt_project"
    )


if __name__ == "__main__":
    dbt_flow_test()

Checklist

  • [x] This pull request references any related issue by including "closes <link to issue>"
    • If no issue exists and your change is not a small fix, please create an issue first.
  • [x] If this pull request adds new functionality, it includes unit tests that cover the changes
  • [x] This pull request includes a label categorizing the change e.g. maintenance, fix, feature, enhancement, docs.

For documentation changes:

  • [x] This pull request includes redirect settings in netlify.toml for files that are removed or renamed.

For new functions or classes in the Python SDK:

  • [x] This pull request includes helpful docstrings.
  • [x] If a new Python file was added, this pull request contains a stub page in the Python SDK docs and an entry in mkdocs.yml navigation.

robfreedy avatar Apr 26 '24 15:04 robfreedy

Deploy Preview for prefect-docs-preview ready!

Name Link
Latest commit 27da33a7751989dc38bcc129c9664a01c9f68e27
Latest deploy log https://app.netlify.com/sites/prefect-docs-preview/deploys/663506addfb9d200080dbd8a
Deploy Preview https://deploy-preview-12964--prefect-docs-preview.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

netlify[bot] avatar Apr 26 '24 15:04 netlify[bot]