dolphinscheduler icon indicating copy to clipboard operation
dolphinscheduler copied to clipboard

[Improvement-#9680][task-plugin] #9680 add hive sql log listener, print MapReduce progress log and obtain application_id allows developers to better track tasks

Open fengjian1129 opened this issue 3 years ago • 14 comments

Purpose of the pull request

Brief change log

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(or)

If your pull request contain incompatible change, you should also add it to docs/docs/en/guide/upgrede/incompatible.md

fengjian1129 avatar Aug 22 '22 10:08 fengjian1129

@EricGao888 hi bro,I have modified the test class, and the local test is available. Please check it。

fengjian1129 avatar Aug 22 '22 10:08 fengjian1129

Codecov Report

Merging #11598 (e2df518) into dev (17a9dd2) will decrease coverage by 0.23%. The diff coverage is 24.32%.

:exclamation: Current head e2df518 differs from pull request most recent head 7124cbd. Consider uploading reports for the commit 7124cbd to get more accurate results

@@             Coverage Diff              @@
##                dev   #11598      +/-   ##
============================================
- Coverage     39.79%   39.55%   -0.24%     
- Complexity     4697     4702       +5     
============================================
  Files           993      998       +5     
  Lines         37662    37994     +332     
  Branches       4186     4241      +55     
============================================
+ Hits          14986    15028      +42     
- Misses        21109    21396     +287     
- Partials       1567     1570       +3     
Impacted Files Coverage Δ
...duler/common/lifecycle/ServerLifeCycleManager.java 0.00% <0.00%> (ø)
...uler/server/master/registry/ServerNodeManager.java 1.18% <0.00%> (-0.02%) :arrow_down:
...server/master/runner/task/CommonTaskProcessor.java 4.54% <0.00%> (-0.22%) :arrow_down:
...lugin/task/api/parameters/DependentParameters.java 0.00% <ø> (ø)
...ache/dolphinscheduler/plugin/task/sql/SqlTask.java 0.00% <0.00%> (ø)
...ver/master/runner/task/DependentTaskProcessor.java 8.26% <9.23%> (+3.00%) :arrow_up:
...nscheduler/service/process/ProcessServiceImpl.java 31.78% <14.28%> (-0.09%) :arrow_down:
...r/api/service/impl/ProcessInstanceServiceImpl.java 60.85% <23.07%> (-0.90%) :arrow_down:
...api/service/impl/ProcessDefinitionServiceImpl.java 32.32% <29.41%> (+0.05%) :arrow_up:
...duler/api/service/impl/WorkerGroupServiceImpl.java 45.34% <30.76%> (-1.33%) :arrow_down:
... and 19 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov-commenter avatar Aug 22 '22 11:08 codecov-commenter

@EricGao888 look this

fengjian1129 avatar Aug 23 '22 02:08 fengjian1129

This is a good feature. Thanks @fengjian1129 for submitting the PR! I add some comments. Could you please also take a look when available? @SbloodyS Thanks!

EricGao888 avatar Aug 23 '22 03:08 EricGao888

To target specific issue from PR, we recommend use keyword fix: #issue_id or close: #issue_id or closes: #issue_id in your PR describe(not title, just desc). It would not only connect issue to PR but also close issue automatically when PR is be closed. @fengjian1129

SbloodyS avatar Aug 23 '22 03:08 SbloodyS

@EricGao888 hi bro, please check

fengjian1129 avatar Aug 23 '22 09:08 fengjian1129

@SbloodyS @zhongjiajie PTAL when available, thanks : )

EricGao888 avatar Aug 26 '22 06:08 EricGao888

@EricGao888 I fixed the code

fengjian1129 avatar Sep 02 '22 03:09 fengjian1129

@EricGao888 I use the result chart to show this change 1.appids write t_ ds_ task_ Instance table and print MR processing log image image image

fengjian1129 avatar Sep 02 '22 04:09 fengjian1129

@EricGao888 t_ds_task_instance table image

fengjian1129 avatar Sep 02 '22 04:09 fengjian1129

@ruanwenjun hi bro, thank you for checking it for me. I have modified the code as required

fengjian1129 avatar Sep 08 '22 11:09 fengjian1129

@caishunfeng Please take a look.

ruanwenjun avatar Sep 13 '22 07:09 ruanwenjun

@ruanwenjun hi bro,review the code for me. thanks

fengjian1129 avatar Sep 15 '22 05:09 fengjian1129

Hi, I would like to ask if the current pr is still going on

MonsterChenzhuo avatar Feb 08 '23 02:02 MonsterChenzhuo

@SbloodyS @ruanwenjun @EricGao888 hi bro. Can you help me to modify this PR and merge it? Because this PR is too far behind.Many friends still want to use this function

fengjian1129 avatar Feb 17 '23 09:02 fengjian1129

This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 7 days if no further activity occurs.

github-actions[bot] avatar Feb 06 '24 00:02 github-actions[bot]

This pull request has been closed because it has not had recent activity. You could reopen it if you try to continue your work, and anyone who are interested in it are encouraged to continue work on this pull request.

github-actions[bot] avatar Feb 13 '24 00:02 github-actions[bot]