dagster icon indicating copy to clipboard operation
dagster copied to clipboard

Add quickstart example for dbt + Dagster

Open maximearmstrong opened this issue 10 months ago • 5 comments

Summary & Motivation

This PR adds a quickstart example for dbt + Dagster. A user that wants to skip the tutorial can copy and paste the code included in the example and run dagster dev.

This example includes the new experimental class DbtProject. Another PR will add more DbtProject examples in the dbt docs and notes that the feature is still experimental.

How I Tested These Changes

BK + local

make mdx-full-format
make next-watch-build

maximearmstrong avatar Apr 16 '24 18:04 maximearmstrong

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @maximearmstrong and the rest of your teammates on Graphite Graphite

maximearmstrong avatar Apr 16 '24 19:04 maximearmstrong

Note that I plan to chime in once the direction of this is a bit more solid.

erinkcochran87 avatar May 01 '24 21:05 erinkcochran87

@rexledesma @tacastillo @cmpadden These are very good points. I think using the scaffold would make sense in the quickstart.

Maybe we could have a quickstart section and another one for the copy-paste code snippets? Or maybe we want to avoid copy+paste code altogether?

The goal of Lopp's internal ticket was to add a section where a user could simply copy+paste code.

@slopp What is your take on this? Would a quick start page with a subset of the tutorial be useful in the context you had in mind? Has this request been made by users?

maximearmstrong avatar May 02 '24 17:05 maximearmstrong

Ah if that's the scope of the original ask, then yeah I'm down for splitting concerns to prevent creep.

will wait for the verdict, but otherwise, feel free to hit re-request review

tacastillo avatar May 02 '24 17:05 tacastillo

Scaffolding is nice and should be encouraged to use, but it sometimes feels heavy to certain new users. But I think having a single file that's copy+paste-able is good - It's handy for us to share how it looks and it's easier for users without a lot of time/patience to digest. So, maybe a path forward is:

  1. Have a dedicated section in the end to show the finished single definitions.py that users can jump directly to and copy+paste if they want to skip the whole setup and explainers. (something like this section but more linkable)
  2. Add a callout at the beginning before "## Step 1: Setup your environment" to the end full example like "skip to the bottom to see the whole thing"
    • something like this: image

Other than that, I agree with all the points made about scaffold and two paths of quickstart.

yuhan avatar May 02 '24 23:05 yuhan

The PR has been updated so that

  • the single file example is at the end of the guide
  • a call to action to the single file example is at the beginning of the guide
    • the user can choose between the scaffold or the copy-paste-able code
  • The quickstart includes how to create a brand new Dagster Project using the scaffold and how to use dagster-dbt in an existing Dagster project
    • see #21816 and #21817

maximearmstrong avatar May 13 '24 18:05 maximearmstrong