dbt-core icon indicating copy to clipboard operation
dbt-core copied to clipboard

[CT-3175] Convert `print` call sites to logging events

Open jtcohen6 opened this issue 1 year ago • 2 comments

Housekeeping

  • [X] I am a maintainer of dbt-core

Short description

There are two places where we currently call print, and should instead fire events. The reason these call print is that we don't want them to include timestamps or other formatting additions when sent to stdout. This enables easy copy-paste or piping to file/jq/etc.

  1. Custom user print Jinja context method (source, docs). Convert to a new UserPrint event.
  2. dbt list with non-JSON logging (source). Convert to existing ListCmdOut event.

We also need a new behavior for the text + stdout logger (and only the combination of text + stdout). We should skip adding formatting (including timestamp/spacing) for the UserPrint + ListOutput events.

Acceptance criteria

  • [ ] Tests for these events with different loggers:
    • stdout + text-formatted → change: no formatting
    • stdout + JSON-formatted → no change
    • log file + text-formatted → no change
    • log file + JSON-formatted → no change
  • [ ] Catalog any remaining instances of Python print method within dbt-core codebase

Impact to Other Teams

  • IDE & CCLI to support custom user-provided print
  • CCLI will also need to implement the same no-formatting logic for UserPrint + ListOutput events

Will backports be required?

Let's just fix this going forward

Context

  • https://github.com/dbt-labs/dbt-core/issues/6543

jtcohen6 avatar Oct 02 '23 19:10 jtcohen6

@jtcohen6 if I understand correctly, this also affects the python CLI? I've been trying to suppress the output of the below command but it seems to ignore the -q and --no-print flags.

command = ["ls", "--models"] + models + ["--output", "json", "--output-keys", "config", "name", "--project-dir", "../../../dbt/", "-q", "--no-print"]
parse_result = dbtRunner().invoke(command)

mrhallak avatar Nov 16 '23 11:11 mrhallak

Relates to #10030

emmyoop avatar May 01 '24 18:05 emmyoop

We should consider deprecating the jinja print in favor of using log when working on this ticket.

ChenyuLInx avatar May 07 '24 14:05 ChenyuLInx

Note: This may also close https://github.com/dbt-labs/dbt-core/issues/9174

graciegoheen avatar May 07 '24 16:05 graciegoheen

There is one potential complication with the dbt list behavior, and it is related to @mrhallak's observation. We don't want to show the dbt version notification or line timestamps, etc., but we do want to show the list output itself. I think that's why we've left this as a print() for so long. So we'll need a plan to deal with that.

peterallenwebb avatar May 07 '24 18:05 peterallenwebb

@peterallenwebb You're right about the need to hide timestamps. We suggest that users who wish to hide unrelated lines use dbt list --quiet, which suppresses all events that aren't "printed" to stdout — the output of dbt list should continue to appear.

Could you imagine adding logic within the EventManager such that:

  • certain events are "print-like" and not suppressed by --quiet
  • those same events should also not be prefixed with timestamps

@dbeatty10 has another good related proposal, which is that --quiet should actually be the default for dbt list (or at least dbt list --output json): https://github.com/dbt-labs/dbt-core/issues/7994

jtcohen6 avatar May 08 '24 11:05 jtcohen6