trino icon indicating copy to clipboard operation
trino copied to clipboard

Add more query, stage, task info in web ui for troubleshooting more conveniently

Open hackeryang opened this issue 10 months ago • 17 comments

Description

  1. Add client info in the query lists of Cluster Overview
  2. Add percentage of CPU, Scheduling, Block Time in Query Details, Live Plan and Stage Performance
  3. Add spilled data size, revokable memory in Query Details and Stage Performance
  4. Add raw input data size skew in Query Details
  5. Add input std dev and input avg in Stage Performance

Additional context and related issues

The following shows some renderings:
image
img_v3_029s_1b7097a1-c5e4-4ad2-a3e7-52f77818a69g
image
img_v3_029s_3228844c-b4ea-488b-8752-b847bf99fb3g

Release notes

( ) This is not user-visible or is docs only, and no release notes are required. (x) Release notes are required. Please propose a release note for me. ( ) Release notes are required, with the following suggested text:

# Section
* Add more query, stage, task info in web ui for troubleshooting more conveniently. ({issue}`21534`)

hackeryang avatar Apr 12 '24 10:04 hackeryang

Add percentage of CPU, Scheduling, Block Time in Query Details, Live Plan and Stage Performance Add raw input data size skew in Query Details Add input std dev and input avg in Stage Performance

  • I would skip std dev and maybe instead display gist of pXX for input rows (see io.trino.operator.OperatorContext#getOperatorMetrics)

Add spilled data size, revokable memory in Query Details and Stage Performance

  • This doesn't display when it's zero, right?

  • Skip blocked % as @raunaqmorarka mentioned

sopel39 avatar Apr 19 '24 16:04 sopel39

@hackeryang is it ready for re-review?

sopel39 avatar May 13 '24 12:05 sopel39

@hackeryang is it ready for re-review?

Sorry not yet, i'm trying to fix this CICD error:
image
But i'm not so good at React JSX, and i didn't figure out a good way to calculate time percents without null or NaN parameters in parseDuration(). Maybe need some advice~

hackeryang avatar May 22 '24 11:05 hackeryang

But i'm not so good at React JSX

That's not my domain either unfortunately.

sopel39 avatar May 24 '24 08:05 sopel39

This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua

github-actions[bot] avatar Jun 14 '24 17:06 github-actions[bot]

Closing this pull request, as it has been stale for six weeks. Feel free to re-open at any time.

github-actions[bot] avatar Jul 08 '24 17:07 github-actions[bot]

Reopening since we still want these changes I assume.

mosabua avatar Jul 08 '24 18:07 mosabua

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot[bot] avatar Jul 14 '24 08:07 cla-bot[bot]

Let us know how you want to proceed @hackeryang .. we are interested to get these changes added from all I can tell.

mosabua avatar Jul 15 '24 19:07 mosabua

Let us know how you want to proceed @hackeryang .. we are interested to get these changes added from all I can tell.

Yes we want to proceed, for now i found a co-author @Max-Cheng who is good at React UI coding, i'll wait him to accomplish the CLA signing then cooperate with him~

@Max-Cheng Please let our community know when you met some problem about CLA signing and separating commits, and please cc the advice above by our reviewers~

Each time you push new commits, remember to rebase our local branch on the latest master branch, otherwise some irrelevant UTs will fail.

And please remember to execute npm install to compile .jsx files into .js files if you have some changes about .jsx: https://github.com/trinodb/trino/blob/master/.github/DEVELOPMENT.md#building-the-web-ui

hackeryang avatar Jul 22 '24 03:07 hackeryang

Let us know how you want to proceed @hackeryang .. we are interested to get these changes added from all I can tell.

Yes we want to proceed, for now i found a co-author @Max-Cheng who is good at React UI coding, i'll wait him to accomplish the CLA signing then cooperate with him~

@Max-Cheng Please let our community know when you met some problem about CLA signing and separating commits, and please cc the advice above by our reviewers~

Each time you push new commits, remember to rebase our local branch on the latest master branch, otherwise some irrelevant UTs will fail.

And please remember to execute npm install to compile .jsx files into .js files if you have some changes about .jsx: https://github.com/trinodb/trino/blob/master/.github/DEVELOPMENT.md#building-the-web-ui

Hello Community. New here.

Finished CLS sign last week, but using the wrong format. I've already sent another email to Martin (I knew that CLA works like batch, not stream ; ) ) So that I can look at this MR later.

I'm happy to help @hackeryang to develop this feature. I'll be back later to finish rebase and answer other reviewers.

Max-Cheng avatar Jul 22 '24 05:07 Max-Cheng

Excellent news @hackeryang and @Max-Cheng

mosabua avatar Jul 22 '24 18:07 mosabua

@cla-bot check

Max-Cheng avatar Jul 23 '24 02:07 Max-Cheng

The cla-bot has been summoned, and re-checked this pull request!

cla-bot[bot] avatar Jul 23 '24 02:07 cla-bot[bot]

Sorry for late with this pull request(Last week on vacation). I'm already finish the

  • Skip blocked % as raunaqmorarka mentioned
  • This doesn't display when it's zero
  • show physicalInputSize for source stage But @hackeryang branch so late compare with upstream master. We will sync two fork repository after than update here.

Max-Cheng avatar Aug 04 '24 12:08 Max-Cheng

@hackeryang please rebase

wendigo avatar Aug 13 '24 12:08 wendigo

I think @wendigo will look at this soon.

mosabua avatar Aug 26 '24 21:08 mosabua