Daft icon indicating copy to clipboard operation
Daft copied to clipboard

feat: Add table viz and persistence to Metrics

Open everettVT opened this issue 1 month ago • 0 comments

Is your feature request related to a problem?

In the current implementation of our new UDF metrics, we currently emit metrics in the swordfish TUI inline with the UDF process. When processing completes and a Dataframe is shown, the metrics are lost from the display.

Image

Then ...

Image

There are a couple of ways this could be accomplished but lets break down the use case and requirements:

  1. Users add metrics to UDFs to gain observability into their processes.
  2. Having live metrics emitted it great, but only if it's also persist-able for further analysis across runs.
  3. Additionally, it's critical that Users are able to see the final metrics for each run at the end.

Describe the solution you'd like

Ideally, we wouldn't lose the metrics and we could somehow display them once the run completed.

The most straightforward way to do this would be by simply printing the metrics in the same manner it is displayed in the TUI as plain text:

╭────────────────────────────────────────┬───────────────────────╮
│ quote                                  ┆ nemotron-response     │
╞════════════════════════════════════════╪═══════════════════════╡
│ I'm going to be the next Hokage!       ┆ **Show:** *Naruto*    │
│                                        ┆ **Character:** N…     │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ I am going to be the king of the pira… ┆ **Show:** One Piece   │
│                                        ┆ **Character:** …      │
╰────────────────────────────────────────┴───────────────────────╯

(Showing first 2 of 2 rows)
16 rows in, 9 rows out, 413 tokens_in,  1,374 tokens_out, 1787 total_tokens, 9 num_requests

Alternatively it would be more more visually appealing if we could just visualize metrics as another table following the results like:

> Results
╭────────────────────────────────────────┬───────────────────────╮
│ quote                                  ┆ nemotron-response     │
╞════════════════════════════════════════╪═══════════════════════╡
│ I'm going to be the next Hokage!       ┆ **Show:** *Naruto*    │
│                                        ┆ **Character:** N…     │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ I am going to be the king of the pira… ┆ **Show:** One Piece   │
│                                        ┆ **Character:** …      │
╰────────────────────────────────────────┴───────────────────────╯
(Showing first 2 of 2 rows)
> Metrics 
╭─────────┬──────────┬───────────┬────────────┬──────────────┬──────────────╮
│ rows_in ┆ rows_out ┆ tokens_in ┆ tokens_out ┆ total_tokens ┆ num_requests │
╞═════════╪══════════╪═══════════╪════════════╪══════════════╪══════════════╡
│ 16      ┆ 9        ┆ 413       ┆ 1374       ┆ 1787         ┆ 9            │
╰─────────┴──────────┴───────────┴────────────┴──────────────┴──────────────╯

Another approach would be to simply label metrics as a map since all metrics are cast to map[str, float]. @srilman visualizing it like you might in excel.

Image

Describe alternatives you've considered

No response

Additional Context

https://github.com/Eventual-Inc/Daft/pull/5549

Would you like to implement a fix?

No

everettVT avatar Nov 12 '25 20:11 everettVT