kyuubi
kyuubi copied to clipboard
[KYUUBI #6107] Collect and summarize the `executorRunTime` and `executorCpuTime` of the statement
:mag: Description
Issue References 🔗
This pull request fixes #6107
Describe Your Solution 🔧
The total execution time of a statement (or a session) is the summary of the execution time of the stages belonging to the statement (or session).
The total execution time of a stage is collected from SQLOperationListener#onStageCompleted
.
The total execution times of the statement or a session are stored in the engine events or output to the log.
Types of changes :bookmark:
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
Test Plan 🧪
Behavior Without This Pull Request :coffin:
Behavior With This Pull Request :tada:
Related Unit Tests
Checklist 📝
- [x] This patch was not authored or co-authored using Generative Tooling
Be nice. Be informative.
Codecov Report
Attention: Patch coverage is 97.87234%
with 1 lines
in your changes are missing coverage. Please review.
Project coverage is 61.13%. Comparing base (
84e60e9
) to head (d9efa2d
). Report is 1 commits behind head on master.
Files | Patch % | Lines |
---|---|---|
...org/apache/spark/kyuubi/SQLOperationListener.scala | 90.90% | 0 Missing and 1 partial :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## master #6112 +/- ##
=========================================
Coverage 61.12% 61.13%
Complexity 23 23
=========================================
Files 623 624 +1
Lines 37195 37237 +42
Branches 5039 5041 +2
=========================================
+ Hits 22737 22766 +29
- Misses 12005 12007 +2
- Partials 2453 2464 +11
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
LGTM overall. Is there any screenshot or logging output to show the results?
LGTM. Thank you for your contribution. Can you update the screenshot to the description of the PR?
done
Thanks. Could you have another try for replacing formatDurationVerbose
with formatDuration
, which is in a shorter form for logging?
Sorry to brother you again.
Thanks. Could you have another try for replacing
formatDurationVerbose
withformatDuration
, which is in a shorter form for logging? Sorry to brother you again.
done
Thanks, merged to master.