dagster
dagster copied to clipboard
Add quickstart example for dbt + Dagster
Summary & Motivation
This PR adds a quickstart example for dbt + Dagster. A user that wants to skip the tutorial can copy and paste the code included in the example and run dagster dev
.
This example includes the new experimental class DbtProject
. Another PR will add more DbtProject examples in the dbt docs and notes that the feature is still experimental.
How I Tested These Changes
BK + local
make mdx-full-format
make next-watch-build
-
#21240
: 3 dependent PRs (#21568
, #21816
, #21817
) 👈
-
#21229
: 1 other dependent PR (#21950
)
-
master
This stack of pull requests is managed by Graphite. Learn more about stacking.
Join @maximearmstrong and the rest of your teammates on Graphite
Note that I plan to chime in once the direction of this is a bit more solid.
@rexledesma @tacastillo @cmpadden These are very good points. I think using the scaffold would make sense in the quickstart.
Maybe we could have a quickstart section and another one for the copy-paste code snippets? Or maybe we want to avoid copy+paste code altogether?
The goal of Lopp's internal ticket was to add a section where a user could simply copy+paste code.
@slopp What is your take on this? Would a quick start page with a subset of the tutorial be useful in the context you had in mind? Has this request been made by users?
Ah if that's the scope of the original ask, then yeah I'm down for splitting concerns to prevent creep.
will wait for the verdict, but otherwise, feel free to hit re-request review
Scaffolding is nice and should be encouraged to use, but it sometimes feels heavy to certain new users. But I think having a single file that's copy+paste-able is good - It's handy for us to share how it looks and it's easier for users without a lot of time/patience to digest. So, maybe a path forward is:
- Have a dedicated section in the end to show the finished single definitions.py that users can jump directly to and copy+paste if they want to skip the whole setup and explainers. (something like this section but more linkable)
- Add a callout at the beginning before "## Step 1: Setup your environment" to the end full example like "skip to the bottom to see the whole thing"
- something like this:
- something like this:
Other than that, I agree with all the points made about scaffold and two paths of quickstart.
The PR has been updated so that
- the single file example is at the end of the guide
- a call to action to the single file example is at the beginning of the guide
- the user can choose between the scaffold or the copy-paste-able code
- The quickstart includes how to create a brand new Dagster Project using the scaffold and how to use dagster-dbt in an existing Dagster project
- see #21816 and #21817